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PREFACE 



Policymakers have long sought credible estimates of the difference federally funded 
employment and training programs make in the employment, earnings, and welfare receipt of those 
they serve. This has proven to be an elusive goal, with problems of research methodology plaguing 
many past efforts to estimate program effects. 

In 1986, the U.S. Department of Labor, following the recommendations of an advisory panel, 
adopted an unusual approach: a study of the impacts of programs funded under the Job Training 
Partnership Act (JTPA), using random assignment, the classical experimental method. The National 
JTPA Study is now producing findings on the services provided by a diverse group of 16 local service 
delivery areas (SDAs) and the impacts these local JTPA programs have on the employment and 
earnings of these they serve. The findings are being issued at an important time, just as amendments 
to JTPA are about to be implemented and interest grows in training and building the quality of the 
workforce. 

This report addresses three topics. First, it explains how the research design that was 
developed to study program impacts was implemented in the participating SDAs - information vital 
to interpreting the results from the study. 

Second, it describes the types of services, such as on-the-job and classroom training, that 
individuals in the study received from JTPA-funded local programs. Service receipt is important 
because it could be a major influence on program impacts. However, the report goes further, 
exploring in some detail how service receipt varied among different subgroups within the overall 
research sample — youths and adult men and women. In doing so, it illuminates a topic of 
considerable public debate in recent years: how JTPA services are allocated among various groups 
that are eligible for the program. 

Finally, the report explores whether differences among study SDAs in the services ihey 
provide and their treatment of different groups are associated with particular characteristics of ihc 
SDAs. It discusses several hypotheses about how labor market conditions, the role of the private 
sector in administration, the sites 9 success in meeting JTPA performance standards, and other factors 
affected the services provided to applicants. 

Initial findings are presented in this and the 18-month impact report that has been produced 
from the study. Further volumes will provide longer-term estimates of program impacts, based on 
30 months of follow-up, and an analysis of the benefits and costs produced by the local JTPA 
programs. Collectively, the reports in the National JTPA Study should provide the type of solid 
information on program effectiveness that has been missing from much of the debate on federally 
funded employment and training programs. 

Judith M. Gueron 
President 

vn 

ERIC J u 



CONTENTS 



PAGE 



THE NATIONAL JTPA STUDY iii 

ACKNOWLEDGMENTS v 

PREFACE vii 

TABLES AND FIGURES » 

ABBREVIATIONS xviii 

EXECUTIVE SUMMARY xix 

CHAPTER 

1 INTRODUCTION 1 

I. An Overview of the Title II-A JTPA System 3 

II. An Overview of the National JTPA Study 7 

III. The Structure of the Study and 

the Content of Its Reports 11 

IV. The Purpose and Organization of This Report , 12 

2 THE STUDY SITES AND THE SAMPLE 15 

I. The 16 Study Sites *. . . 17 

II. The Sample for This Report 37 

3 BALANCING RESEARCH GOALS AND THE 

REALITY OF JTPA PROGRAMS 45 

I. The Goals of the National JTPA Study 45 

II. Client Flow and Study Procedures 46 

4 KEY POLICY QUESTIONS AND THE RESULTING STUDY DESIGN 60 

I. Choosing the Random-Assignment Design 60 

II. Providing Alternative Program 

Impact Estimates 64 

III. Defining Understandable and 

Operationally Relevant Service Strategies 67 

5 SERVICES RECEIVED BY ADULT ENROLLEES 78 

I. Service Receipt in the 

Five Service Categories SO 

II. The Length of Enrollment for Adults 96 



ix 



ERLC 



11 



CHAPTER 



PAGE 



6 SERVICES RECEIVED BY OUT-OF-SCHOOL YOUTH ENROLLEES 103 

I Service Receipt in the 

Five Service Categories *°* 

H. The Length of Enrollment 

for Out-of-Schooi Youths 120 

7 VARIATIONS ACROSS THE SERVICE DELIVERY AREAS 126 

I An Overview of the Findings 130 

IL Variation in Service Strategy Recommendations, 

Population Served, and Services Received 132 

IIL The Relationship Between SDA Characteristics 

and Service Emphasis and Population Served 144 

APPENDICES 

Appendix A. Profiles of the Service Delivery Areas in the 

National JTPA Study 165 

Appendix B. Supplemental Tables to 

Chapters 4, 5, and 6 191 

Appendix C Data Sources for This Report 239 

REFERENCES 243 

SELECTED MDRC PUBLICATIONS 245 



X 

!>2 



TABLES AND FIGURES 



TABLE PAGE 

1 Sites in the National JTPA Study xxv 

2 Enrollment Rates for Treatment Group Members, 

by Target Group and Service Strategy ♦ xxviii 

3 Key Services for Enrollees, 

by Target Group and Service Strategy , xxxi 

4 Service Receipt Rates for Adult and Out-of-School 

Youth Enrollees, by Selected Baseline Characteristics xoriv 

2.1 Key Facts About the National JTPA Study Sites 18 

2.2 Local Population and Economic Characteristics, by Site 21 

23 Organizational Structure of the SDAs Participating 

in the National JTPA Study, by Site 24 

2A Major Types of Service Providers Used by Sites 
Participating in the National JTPA Study, 

by Selected Program Services 25 

2.5 Roles of SDAs and Service Providers at Selected Points 

in the Client Flow, by Site 28 

2.6 Level of Use of Performance-Based Contracts Among 

Sites in the National JTPA Study, for Adults and Youths 30 

2.7 Selected Characteristics of JTPA Title II-A Terminees, 

by Site 31 

2.8 Selected Characteristics of JTPA Title II-A Terminees 

That Are Obstacles to Employment, by Site 32 

2.9 Selected Characteristics of the JTPA Title II-A Programs 

at the National JTPA Study Sites 34 

JTPA Program Outcomes for Adults, by Site 35 

2.1J XfPA Program Outcomes for Youths, by Site 36 

2.12 Percentage Distributions of Selected Baseline 
Characteristics of Treatment Group Members, 

by Target Group 38 

xi 



ERLC i 3 



TABLE 



PAGE 



2.13 Percentage Distributions of Employment Barrier 

Subgroups for Treatment Group Members, by Target Group 41 

2.14 Percentage Distributions of Employment Barrier 
Subgroups for Treatment Group Members, 

by Selected Baseline Characteristics and Target Group 42 

3.1 Program Services Available in the Three Service Strategies 53 

3.2 Percentage Distribution of Post-Random Assignment 
Activity in JTPA of Treatment Group Members 

Who Did Km Enroll 58 

4.1 Percentage Distributions of Service Strategy 
Recommendations and Recommended Program Services, 

by Target Group 68 

4.2 Enrollment Rates for Treatment Group Members, 

by Service Strategy and Target Group . 70 

43 Key Services for Treatment Group Members 

and Enrollees, by Target Group and Service Strategy 71 

4.4 Service Receipt Rates for Treatment Group Members 

and Enrollees, by Service Strategy 73 

4.5 Percentage Distributions of Post-Random Assignment 
Activity in JTPA of Treatment Group Members 

Who Did Not Enroll, by Service Strategy 76 

5.1 Service Receipt Rates for Adult Enrollees, 

by Selected Baseline Characteristics 81 

5.2 Service Receipt Rates for Adult Enrollees, 

by Employment Barrier Subgroup and Target Group 89 

53 Service Receipt Rates for Adult Enrollees, 

by Employment Barrier Subgroup and Ethnicity 92 

5.4 Median Length of Enrollment for Adults, by Selected 

Baseline Characteristics and Service Receipt 97 

6.1 Service Receipt Rates for Out-of-School Youth 

Enrollees, by Selected Baseline Characteristics 106 

6.2 Service Receipt Rates for Out-of-School Youth 
Enrollees, by Employment Barrier Subgroup 

and Target Group 113 



xii 



ERLC 



14 



TABLE 



PAGE 



63 Service Receipt Rates for Out-of-School Youth 
Enrollees, by Employment Barrier Subgroup 

and Ethnicity 116 

6.4 Median Length of Enrollment for Out-of-School Youths, 

by Selected Baseline Characteristics and Service Receipt 121 

Tl Percentage Distributions of Service Strategy 

Recommendations for Adult and Youth Treatment Group Members, 
by Site, in Order of Highest to Lowest Percentage 

Recommended for OJT0SA 133 

7.2 Percentage of Adult Treatment Group Members 

in Employment Barrier Subgroups and Their Enrollment 
Rates, by Site, in Order of Highest to Lowest Enrollment 

Rate Difference 136 

73 Percentage of Youth Treatment Group Members 

in Employment Barrier Subgroups and Their Enrollment 
Rates, by Site, in Order of Highest to Lowest Enrollment 

Rate Difference 138 

7.4 Service Receipt Rates for Adult and Youth Enrollees, by Site, 

in Order of Highest to Lowest Percentage Receiving OJT 142 

7.5 Percentage Distributions of Service Strategy 
Recommendations for Adult and Youth Treatment Group Members, 
by Site Achievement of the JTPA Performance Standard 

for Entered Employment 148 

7.6 Percentage Distributions of Service Strategy 
Recommendations for Youth Treatment Group Members, 
by Site Achievement of the JTPA Performance Standard 

for Youth Positive Terminations 149 

7.7 Percentage Distributions of Service Strategy 
Recommendations for Adult and Youth Treatment 

Group Members, by SDA Organizational Structure 152 

7.8 Percentage of Adult and Youth Treatment Group Members 
in Employment Barrier Subgroups and Their Enrollment 
Rates, by Site Achievement of the JTPA Performance 
Standards for Entered Employment and Youth Positive 

Terminations 154 

7.9 Percentage of Adult and Youth Treatment Group Members 

in Employment Barrier Subgroups and Their Enrollment Rates, 

by SDA Organizational Structure 157 



xui 



ERLC 



TABLE PAGE 

7.10 Percentage of Adult and Youth Treatment Group Members 

in Employment Barrier Subgroups and Their Enrollment Rates, 

by Site Level of Use of Performance-Based Contracts 160 

Bl Percentage Distributions of Service Strategy 

Recommendations for Adult Male Treatment Group Members, 

by Selected Baseline Characteristics 192 

B.2 Percentage Distributions of Service Strategy 

Recommendations for Adult Female Treatment Group Members, 

by Selected Baseline Characteristics 194 

B3 Percentage Distributions of Service Strategy 

Recommendations for Male Youth Treatment Group Members, 

by Selected Baseline Characteristics 1% 

B.4 Percentage Distributions of Service Strategy 

Recommendations for Female Youth Treatment Group Members, 

by Selected Baseline Characteristics 198 

E-5 Enrollment Rates for Adult Male Treatment Group 
Members, by Selected Baseline Characteristics and 

Service Strategy 200 

B.6 Enrollment Rates for Adult Female Treatment Group 
Members, by Selected Baseline Characteristics and 

Service Strategy 202 

B.7 Enrollment Rates for Male Youth Treatment Group 
Members, by Selected Baseline Characteristics and 

Service Strategy 204 

B.8 Enrollment Rates for Female Youth Treatment Group 
Members, by Selected Baseline Characteristics and 

Service Strategy 206 

B.9 Service Receipt Rates for Adult Enrollees, by Number 

and Type of Services Received and by Target Group 208 

B.10 Service Receipt Rates for Out-of-School Youth Enrollees, 

by Number and TVpe of Services Received and by Target Group 209 

B.ll Service Receipt Rates for Adult Male Enrollees, 

by Selected Baseline Characteristics 210 

B.12 Service Receipt Rates for Adult Female Enrollees, 

by Selected Baseline Characteristics 212 



xiv 




TABLE PAGE 

B.13 Service Receipt Rates for Male Youth Enrollees, 

by Selected Baseline Characteristics 214 

B.14 Service Receipt Rates for Female Youth Enrollecs, 

by Selected Baseline Characteristics 216 

B.15 Service Receipt Rates for Adult Male Enrollecs, 

by Employment Barrier Subgroup and Ethnicity 218 

B.16 Service Receipt Rates for Adult Female Enrollecs, 

by Empoyment Barrier Subgroup and Ethnicity 220 

B.17 Service Receipt Rates for Male Youth Enrollees, 

by Employment Barrier Subgroup and Ethnicity 222 

B.18 Service Receipt Rates for Female Youth Enrollees, 

by Employment Barrier Subgrcup and Ethnicity 224 

B.19 Median Length of Enrollment for Male Adults, by 

Selected Baseline Characteristics and Service Receipt 226 

B.20 Median Length of Enrollment for Female Adults, by 

Selected Baseline Characteristics and Service Receipt 229 

B.21 Median Length of Enrollment for Male Youths, by 

Selected Baseline Characteristics and Service Receipt 232 

B.22 Median Length of Enrollment for Female Youths, by 

Selected Baseline Characteristics and Service Receipt 235 

B.23 Enrollment Rates for Control Group Members, by Service 

Strategy and Target Group 238 



ERLC 



XV 

17 



FIGURE 



PAGE 



1 Random Assignment Model for the National JTPA Study sriii 

2.1 Location of the National JTPA Study Sites 19 

3.1 Flow cf Sample Members in the National JIPA Study Sites 47 

4.1 Potential Design for Random Assignment to a 
Service Strategy or to the Control Group 

(Not Used in the National JTPA Study) 61 

4.2 Final Design for Random Assignment Following 
Assessment and Recommendation of a Service Strategy 

(Used in the National JTPA Study) 63 

7.1 Factors Affecting the Implementation of JTPA Programs 127 



xvi 

J 8 



ABBREVIATIONS 



ABE 

AFDC 

BE 

BTF 

CETA 

CT 

ct-os 

DOL 

ESL 

GED 

JASR 

JSA 

JTLS 

JTPA 

JTQS 

MDRC 

mis 

NORC 

OJT 

OMB 

PIC 

RFP 

SDA 

WIN 



adult basic education 

Aid to Families with Dependent Children 

basic education 

Background Information Form 

Comprehensive Employment and Training Act (1973) 

classroom training 

classroom training in occupational skills 
U.S. Department of Labor 
English as a second language 

General Educational Development certification (high school equivalency) 

JTPA Annual Status Report 

job search assistance 

Job Training Longitudinal Survey 

Job Training Partnership Act (1982) 

Job Training Quarterly Survey 

Manpower Demonstration Research Corporation 

management information system 

National Opinion Research Center 

on-the-job training 

Office of Management and Budget 

private industry council 

Request for Proposals 

service delivery area 

Work Incentive Program 



xvui 



9 

ERIC 



hi 



EXECUTIVE SUMMARY 



The Job Training Partnership Act of 1982 (JTPA) funds the major federal program 
providing job training and other employment-related services for economically disadvantaged 
individuals. The act established more than 600 local service delivery areas (SDAs), which play 
a central role in designing and delivering services under Title Et-A, the main title of the JTPA 
legislation. As part of the move toward greater accountability in the funding of social 
programs, JTPA calls for a system of state-administered performance standards for local 
programs, and for monitoring of program operations, with the goal of determining the 
program's success in increasing the employment and earnings and reducing the welfare receipt 
of the people it serves. 

The National JTPA Study is the Department of Labor's (DOL's) major effort to assess 
the impacts and cost-effectiveness for adults and out-of-school youths of programs funded under 
Title II-A in a diverse sample of 16 SDAs across the country. Following the recommendations 
of a technical advisory panci established to assist in studying JTPA's impacts, DOL chose to 
conduct a Geld experiment in which JTPA applicants whom local staff judged eligible and 
appropriate for the program were randomly assigned to a treatment (or program) group, which 
was given access to JTPA-funded services, or a control group, which was not DOL chose this 
approach, despite the challenges it would present, because of the inability of past non-random- 
assignment impact studies to provide credible estimates of program impacts. This is the first 
random-assignment impact study of local programs in an ongoing, national system of 
employment and training services that serves primarily volunteers. 

The National JTPA Study has two major parts, which were awarded as separate, 
competitively bid contracts. Under one contract, the Manpower Dem onstration Research 
Corporation (MDRC) — with Abt Associates Inc. as a subcontractor — recruited sites, 
monitored the implementation of the research design, and described and analyzed the 
characteristics of the local programs in the study and participation in JTPA services, a task 
completed in this report Under a second contract, Abt Associates Inc. — with ICF, MDRC, 
New York University, and National Opinion Research Center (NORC) as subcontractors - 
collects baseline and follow-up information on those randomly assigned and conducts the impact 
and benefit-cost analysis, while also using the framework of an experiment to search for better 
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alternative methods for estimating program impacts. The first impact report analyzes the 
interim impact findings based on 18 months of post-random-assignment follow-up, 1 and a later 
report will present 30-month impact findings. 

A Preview of the Findings 

Several basic findings emerge from this report The first three relate to the overall study 
and sample and the last two concern differences among groups within the overall sample or 
among study sites. First, on many counts, the 16 study sites, as a group, resemble the national 
average for all JTPA programs; they include much of the diversity of the national system 
Respite the fact that they were not selected randomly from all SDAs. Second, these sites 
successfully implemented a complex research design, randomly assigning more than 20,000 
applicants to treatment and control groups. As part of this design, local site staff designated 
one of three service strategies for each person randomly assigned. Third, although not all 
treatment group members enrolled in the services recommended for them, these service strategy 
recommendations proved to have clear links to the types of services in which members of the 
treatment group actually enrolled. Thus, treatment group members recommended for the 
classroom training service strategy received services with a strong classroom focus, while most 
of those who were recommended for the on-the-job-training/job search assistance (OJT/JSA) 
service strategy, and who enrolled in JTPA, were active in services designed to find them an 
OJT (subsidized) position or regular employment (All of those in the OJT/JSA group had 
been recommended for OJT in case that were to prove necessary as a bridge to regular 
employment) The third strategy - by design, a mixture of services - emphasized specific 
services for adults and youths, as discussed later in this summary. Because these initial 
recommendations translated into distinct emphases in program sendees, the separate impact 
estimates for the individuals recommended for these three strategies (presented in the 18- 
month impact report) have an operational meaning for local programs. 

Fourth, there were differences in the types of services in which the four target groups 
in the study (adult men, adult women, male out-of-school youths, and female out-of-school 



1 See Howard S. Bloom, Larry L. Orr, George Cave, Stephen H. Bell, and Fred Dooiittle, The 
National JTPA Study: Title IUA Impacts on Earnings and Employment at 18 Months (Bethesda, Md.: 
Abt Associate Inc., 1993). 
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youths) enrolled and further differences among other key subgroups. In general, both male 
adults and male youths were more likely to enroll in OJT and job search assistance than their 
female counterparts, who had higher rates of enrollment in classroom training in occupational 
skills (CT-OS). Among adults, blacks and Hispanics were more likely than whites to enroll in 
CT-OS and less likely to enroll in OJT; among youths, the cnty consistent difference was that 
whites were more likely than blacks and Hispanics to enroll in OJT. Some of these differences 
are related to the extent that each group tends to have barriers to employment such as low 
educational attainment, limited work experience, or welfare receipt Finally, there is 
considerable variation in site administrative practices (e.g., the role of the private industry 
council and the use of performance-based contracts), success in exceeding performance 
standards, service emphasis, and patterns of enrollment for treatment group members. For 
example, sites varied greatly in their emphasis on OJT versus CT-OS and their propensity to 
enroll especially disadvantaged applicants. The analysis provides suggestive evidence of the 
relationships between these differences and administrative practices or local conditions. 

An Overview of the National XTPA Study 

The central goal of the study is to understand, in a diverse sample of sites, the difference 
that services funded under Title II-A of JTPA make in the employment, earnings, and welfare 
receipt of adults and out-of-school youths served by the program: i.e., the program's impacts. 
This information will tell policymakers and program operators how well program participants 
fare in the labor market over and above what they would have dono on their own, without 
access to the program. Evaluation experts and policymakers have increasingly come to the 
conclusion that the most reliable studies of program impacts use random assignment - a 
process like a drawing or lottery, in which eligible applicants for a program are randomly 
assigned to a group to be served (the "treatment" or "program" group) or to a group not given 
access to the program (the "control" group). Because the two groups are created randomly, 
there is no systematic difference between them prior to random assignment, and the labor 
market experiences of the control group provide a benchmark against which to compare the 
experiences of the treatment group. 

In the National JTPA Study, DOL wanted to estimate overall impacts for the Title II- 
A JTPA program for adults and out-of-school youths in participating sites and specific impacts 
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for other subgroups to the extent that sample sizes allow. In addition/ DOL also initially called 
for separate impact estimates for individuals recommended for key services such as CT-OS, 
OJT, and job search* As the study was implemented, three distinct service strategies emerged, 
for which separate impact estimates are calculated: (1) training to build skills through 
classroom-based services (labeled "classroom training"); (2) subsidized OJT (for which all in the 
group were recommended) and/or job search assistance to find regular employment (labeled 
"OJT/JSA"); and (3) a residual category (labeled "other services"), which could involve various 
combinations of other education and employment-related services. This third strategy allowed 
the study to include the full range of program services. Finally, DOL wished the study to be 
conducted in a manner that would change normal operations as little as possible. 

Together, these goals created tensions in the research design. Impact estimates of the 
entire JTPA program in participating sites would call for random assignment early in the intake 
process to assure that the control group would receive little or no JTPA assistance. However, 
the desire to produce impact estimates for specific service strategies and to allow normal staff 
decisions on service recommendations would call for random assignment after local staff 
assessed the skills, interests, and service needs of applicants, though before actual services 
began. And defining service strategies that are clearly distinguished from each other (e.g., 
classroom-based services versus employment-based services) proved a challenge because of the 
diversity of local services that could be used singly or in combination. 

The resulting research design is shown in Figure 1, which superimposes the special 
research procedures on the normal steps by which clients apply for and enroll in JTPA. Local 
sites and service providers recruited applicants from among the eligible population in their 
community (Step #1 in Figure 1). JTPA funding is typically sufficient to serve about 10 
percent or fewer of those eligible, and in the study sites — as is the case nationally — a 
substantial proportion of those who initially inquired about JTPA services never reached the 
stage of enrollment in the program. Individuals who pursued the opportunity for JTPA services 
completed an application form to establish eligibility (Step #2), participated in an assessment 
of their service needs (Step #3), and (in one of the special steps added for the study) 
completed an Informed Consent Form and a Background Information Form to provide 
information on their pre-random-assignment characteristics (Step #4). Local staff then 
recommended specific JTPA services for the applicants (or indicated that JTPA was not 



xxii 



appropriate) and designated a service strategy for each appropriate client (Step #5), which the 
staff were to follow in working to arrwige services if the person were randomly assigned to the 
treatment group. Within each service strategy, clients were then randomly assigned (Step #6) 
to a treatment group, whose members were given access to the JTPA-fundcd services, or to 
a control group, whose members could not participate in JTPA-funded services for 18 months 
but who were given a list of other services in the community. Staff then worked with 
treatment group members to arrange the recommended services (Step #7) and enroll in JTPA 
those for whom they were successful. 

Under this design, the overall impact of JTPA services in the participating sites is 
estimated by combining all three treatment groups and comparing their experiences with those 
of a combined control group. The impacts of each of the service strategies for those referred 
to these services are estimated by comparing the treatment group within the service strategy with 
its control group. However, since different types of clients are recommended for the three 
service strategies, it is not possible to directly compare experimentally the impacts of the service 
strategies; since both the services and the initial characteristics of the clients would vary across 
the strategies, it is difficult to isolate the difference that the choice of service strategy made. 

Findings op the Sites in the Study 

• On many counts, the 16 study sites as a group are similar to the national 
average for all SDAs. 

Although the 16 sites listed in Table 1 were not chosen randomly from all SDAs in the 
country, their economic and labor market conditions, poverty rates, client characteristics, and 
performance as measured by DOL standards are similar to the national average for all SDAs 
during the late 1980s, when random assignment was under way. For example, the averages for 
the study sites and for all SDAs nationally were virtually the same for poverty rates (about 10 
percent), unemployment during the period of random assignment (6.6 percent), and per capita 
annual earnings of workers in the SDA ($18,100). Of the people served by the study sites and 
all SDAs nationally, about 60 percent were white, slightly more than 25 percent were black, 
and 10 percent were Hispanic. The average percentage of those served in the study sites who 
had one of a variety of obstacles to employment - e.g., low educational attainment, receipt of 



xxiv 



Aid to Families with Dependent Children (AEDC), or limited work experience - was similar 
to the average percentage of those with these barriers in all SDAs. 



TABLE 1 



SITES IN THE NATIONAL JTPA STUDY 



Butte, Moot 
Cedar Rapids, Iowa 
Coosa Valley, Gil 
Corpus Christi, Tex. 
Decatur, HL 
Fort Wayne, Ind- 
Heartland, Fla. 
Jackson, Miss. 



Jersey City, N J. 
Larimer County, Colo. 
Marion, Ohio 
Northwest Minnesota 
Oakland, Calif. 
Omaha, Neb. 
Providence, ILL 
Springfield, Mo. 



Summary measures of program operation also show strong similarities. Average lengths 
of JTPA enrollment for the study sites and all sites nationally were within a week of each 
other, though average program costs per adult served in the study sites were 6 percent above 
those for all sites. Importantly, on key JTPA performance measures, the study sites exceeded 
their standard by about the same (or a slightly smaller) percentage all SDAs nationally. For 
example, study sites exceeded their adult job placement standard by an average of 6 percentage 
points in program year 1988 (the year when the bulk of the sample was enrolled) compared 
with 7 percentage points for all sites nationally. Thus, the sites as a group are neither at the 
high nor low end of the performance "scale." 

Behind these averages, however, there is great variation in the sites* local conditions, 
clients, program design, administrative practices, and performance, as is the case nationally. 
The sites include some with low unemployment and strong economic growth and some in 
declining areas; sites in large metropolitan areas and sites that are entirely rural; and ethnic 
mixtures that range from virtually all black and/or Hispanic to almost entirely white. During 
the late 1980s, the average length of JTPA enrollment in the study sites ranged from less than 
two months to nearly eight months for adults, and slightly more than one month to nearly eight 
months for youths. On performance standards, there was equally wide variation: some sites 
greatly exceeded their standards, while others failed to meet them. Again, this suggests that 
while the study sites are not representative in the statistical sense, they do include much of the 
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diversity of the national JTPA system. The major exception is that they do not include any 
of the nation's largest cities. 

Findings on the Implementation of Random Assignment 

• The 16 sites successfully implemented a complex research design, with more 
than 20,000 individuals randomly assigned over a 23-month period 

Normally, successful implementation of a research design does not qualify as a major 
finding. The National JIPA Study, however, is among the most complex undertaken in the 
employment and training field, and successful implementation of study procedures by sites was 
not a certainty. The 16 sites randomly assigned 20,601 individuals between November 1987 and 
September 1989. The sample analyzed in this report and the 18-month impact report includes 
17,031 individuals randomly assigned before July 1989. 

Virtually none of the control group participated in the JTPA program in the study sites. 
Site staff were able to put in place an administrative system that allowed them throughout the 
follow-up period to identify applicants randomly assigned to the control group and to avoid 
enrolling them in the JTPA-funded services under study. Only 3 percent of the control group 
was enrolled in JTPA-funded services at any point during the 18 months following random 
assignment, the period during which they were excluded from such services in the study sites. 
Thus, the intended difference in access to JTPA services between the treatment and control 
groups emerged. This level of site compliance with study procedures is especially impressive 
given the many organizations that can be involved in client recruiting, applications, assessment, 
and service delivery in local JTPA programs. 

• Sites did have to expand their recruitment of applicants to permit creation 
of a control group, but eligibility determination, assessment, and service 
delivery were not, in general, affected by the study. 

Creation of a control group, coupled with a declining unemployment rate over the course 
of random assignment, led to increasingly severe recruitment problems in many sites. Site staff 
reported that these two factors led them to recruit and serve less job-ready clients than they 
had during the recession of the early and mid 1980s, when many individuals with extensive 
work experience and job-related skills applied for JTPA services. 
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Hcvever, in the subsequent steps of client flow through the JTPA program (Le., eligibility 
determination, assessment, and service delivery), the study did not lead to substantial changes 
in local operations, other than the need to increase the number of applicants who proceeded 
to the stage of random assignment Local staff determined eligibility and appropriateness for 
JTPA — following normal procedures — and recommended appropriate services. These service 
recommendations were used to designate one of three broadly defined service strategies for 
each individual 

This finding that the study itself did not lead to significant changes in the later stages of 
program intake does not mean that local administrative practices remained unchanged. 
Nationally, in the late 1980s, JTPA was becoming increasingly concerned about the need to 
serve less job-ready clients, provide more intensive services, improve recruitment practices, and 
introduce greater accountability in contracting procedures. Not surprisingly, in light of this 
evolution of the system, several of the study SDAs on their own initiative made major reforms 
in their procedures during the study. Because the goal of the research was to study JTPA 
programs that would operate as normally as possible, there was no attempt to "freeze" program 
operations into the forms they had when sites entered the project 

Findings on JTPA Enrollment for the Treatment Group 

The tensions among the research goals discussed earlier led to a research design in which 
random assignment occurred when local staff recommended applicants for various possible 
services rather than at the point when major services were about to begin. As discussed 
earlier, this choice was made to allow staff to assess clients as they normally would and to 
identify the control group before referrals to services began. Following random assignment, 
local staff worked with members of the treatment group to arrange the desired services, but 
they were not successful in all cases. Some treatment group members lost interest in the 
program, found a job on their own, or moved, while for others staff could not find a service 
provider or employer. 

• Sixty-four percent of the treatment group sample enrolled in JTPA during the 
18-month follow-up period. There were consistent differences in enrollment 
rates across the service strategies, with classroom training having the highest 
enrollment rate aud OJT/JSA the lowest Differences in target group 
enrollment rates within service strategies were relatively small. 
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Table 2 presents JTPA enrollment rates by service strategy and target group, with the 
service strategies showing much greater variation than the target groups. For the service 
strategies, enrollment ranged from a low of 57 percent for OJT/JSA to a high of 72 percent 
for classroom training. Within service strategies, there was little variation by target group, 
though male youths had the highest enrollment rate in all three strategies. Enrollment rates 
for the key target groups varied from 61 percent for adult men to 67 percent for male out- 
of-school youths. 



TABLE 2 

ENROLLMENT RATES FOR TREATMENT GROUP MEMBERS, 
BY TARGET GROUP AND SERVICE STRATEGY 



Target Group 


All 
Strategies 

(%) 


Classroom 
Training 
(%) 


onyjSA 
(%) 


Other 
Services 

(%) 


All Target Groups 


63.8 


72.4 


565 


623 


Adult Men 


608 


112 


56.6 


58.9 


Adult Women 


64.6 


72* 


55.4 


6X4 


Male Youths 


66.8 


74* 


5S5 


67.7 


Female Youths 


655 


115 


S75 


63.1 



SOURCE: MDRC calculations from Background Information Form responses and program enrollment data from 
the 16 SDAs. 



The differences in enrollment rates can be at least partly explained by differences in the 
difficulty of arranging the services in each strategy. While arranging classroom training is not 
necessarily easy, developing OJT positions often is more difficult. In most SDAs, especially 
those in which there is a large city, there are several agencies whose mission is to provide 
classroom training, increasing the chances that a match can be made between a client's needs 
and interests and the courses offered. However, to develop an OJT position, JTPA staff enlist 
the cooperation of private or nonproflt employers whose main goal is to produce a good or 
service and not to provide training for the economically disadvantaged. Staff must rely on a 
combination of financial incentives and assurances of a low administrative and supervisory 
burden to recruit employers and typically contact many firms before a position can be found. 
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At the same time, the client may be actively seeking a job through other means and may find 
employment without ever being placed in a JTPA-funded OJT. Hence, the lower JTPA 
enrollment rate in the OJT/JSA service strategy is not surprising. 

• About half of the treatment group members who were never enrolled in JTPA 
probably received some type of post-random-assignment service from the 
program. 

Many local programs in the study did not enroll applicants in JTPA until they were active 
in services intended to increase their employability. In the JTPA system, and especially the 
performance standards, there were incentives to delay enrolling applicants. JTPA performance 
standards in place during the period of random assignment emphasized (1) achieving a high 
proportion of "success stories'* for enrollees, where success is defined as placement in a job that 
pays well or achievement of a "positive termination" for youths, and (2) keeping costs per 
success story low. Since only enroliees counted in the sample for which performance was 
measured, and many local programs believed that they had discretion in defining when 
enrollment occurred, it was common for SDAs to delay enrolling clients in JTPA. For 
example, clients seeking OJT would not be enrolled in JTPA until staff were able to arrange 
an OJT position with an employer and the client showed up for the first day of work. Unless 
these efforts to arrange a service were successful, the client might not be enrolled in JTPA. 

Following random assignment, local staff lost contact with some members of the treatment 
group or found that they were no longer interested in the program. In other cases, staff were 
in contact with applicants, but did not arrange any type of referral to a possible service 
provider. However, a special study using a small sample of nonenrolled treatment group 
members showed that about half of the 36 percent of treatment group members who were 
never enrolled in JTPA did receive some services from the program. In most cases, this was 
a referral to a possible OJT position or participation in a job club, so this nonenrolled 
"participation" in JTPA was most common in the OJT/JSA service strategy. Overall, about 80 
percent of the treatment group (the 64 percent who were enrolled and an additional 18 
percent who were not) had some involvement with local JTPA staff after random assignment. 
Therefore, the proportion of the treatment group for whom local JTPA staff tried to arrange 
services is greater than the JTPA enrollment rate because of site practices adopted in response 
to performance standards. 
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Findings on the Link Between Recommended Service Strategics 
and Enrollment in Particular Services 

One central goal of the study is to calculate impact estimates for individuals 
recommended for three service strategies. For this effort to be successful, the services received 
by individuals referred to these three strategies have to be clearly distinguishable in emphasis 
and reflr zt important service approaches commonly used in JTPA. One of the uncertainties 
about the research design of the study was whether individuals recommended fc r each of the 
service strategies would actually be enrolled in these services. 

This section explains what these service strategies mean by describing the services 
received by individuals in the three groups. These findings are summarized in Table 3, which 
presents the two key services in which JTPA enrollees were active and the proportion of the 
relevant sample enrolled in them, for each target group/service strategy combination. 

• Most treatment group members recommended for the classroom training . 
service strategy were enrolled in classroom-based services - CT-OS or basic 
education. 

All treatment group members in this service strategy were recommended for CT-OS, 
though other services such as basic education or job search assistance may also have been 
recommended. As mentioned above, SDA staff did not usually encounter major problems 
arranging classroom-based services for JTPA applicants. This is reflected in the fact that 72 
percent of the treatment group members recommended for the classroom training service 
strategy were enrolled in JTPA (Table 2), and at least 80 percent of these enrollees in each 
target group were active in either CT-OS, basic education, or both (Table 3). Eighty-six 
percent of adult men, 89 percent of adult women, 80 percent of male youths, and 86 percent 
of female youths were active in either CT-OS or basic education. 

• JTPA enrollees in the OJT/JSA service strategy were split equally between on- 
the-job training services and job search assistance. This reflects the fact that 
treatment group members recommended for the OJT/JSA service strategy were 
oriented toward immediate employment, either through subsidized training on 
the job or an unsubsidized position. 

Individuals recommended for this service strategy were usually interested in immediate 
employment because of their financial needs or lack of interest in classroom-based services. 
However, local staff felt that they may have needed training to improve their skills in order to 



TABLE 3 

KEY SERVICES FOR ENROLLEES, 
BY TARGET GROUP AND SERVICE STRATEGY 



Target Group 



Classroom 
Training 



OJT/JSA 



Other 
Services 



Adult Men 



86% enrolled in 
CT-OS or BE 



87% enrolled in 
OJTor JSA 



89% enrolled in 
JSA or misc. services 



Adult Women 



89% enrolled in 
CT-OS or BE 



88% enrolled in 
OJTor JSA 



82% enrolled in 
JSA or misc. services 



Male Youths 



80% enrolled in 
CT-OS or BE 



85% enrolled in 
OJTor JSA 



83% enrolled in 
BE or misc. services 



Female Youths 



86% enrolled in 
CT-OS or BE 



85 % enrolled in 
OJTor JSA 



80% enrolled in 
BE or misc. services 



SOURCE: MDRC calculations from Background Information Form responses and from program 
enrollment and participation data from the 16 SDAs. 
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find long-term employment and, therefore, recommended OJT as an option for all in this 
group. Fifty-seven percent of treatment group members recommended for the OJT/JSA service 
strategy enrolled in JTPA, and at least 85 percent of these enrollees in each target group 
received employment-based services that included OJT, job search assistance, or both. 

The observed split between OJT and job search occurred because, for OJT, staff had to 
find an employer interested in hiring a new worker. In some cases, staff would have to 
arrange subsidized OJT to persuade the employer to hire an applicant, while in other cases 
they would not The mixture of enrollment in OJT and job search assistance among the 
treatment group in the OJT/JSA service strategy reflects the result of staff's efforts to find 
employment for these clients. 

9 The diverse needs of the clients recommended for the other services strategy 
are reflected in the wide range of JTPA services that they received. For 
adults, the key services were Job search assistance and miscellaneous services 
(usually preemployment skills); for youths, the most common services were 
basic education and miscellaneous services. 

The creation of the other services strategy enabled SDA staff to accommodate clients 
with a wide range of needs, including some of the most job-ready clients, who were 
recommended for job search assistance only, as well as some of the least job-ready clients, who 
were recommended for basic education as a first service, with subsequent steps uncertain. It 
also included adult clients who were recommended for a variety of preemployment skills 
programs or for combined OJT and classroom training and youths who were recommended for 
special Title H-A-funded programs designed specifically for school dropouts. 

For adults, both men and women, the two key services within this strategy were 
miscellaneous services (ranging from further assessment and preemployment skills preparation 
to work experience and special introductions to the world of work) and job search assistance. 
Among adults who enrolled in JTPA, 89 percent of the men and 82 percent of the women 
received one or both of these services. For youths, basic education and miscellaneous services 
were the key services, with 83 percent of male youths and 80 percent of female enrollees 
receiving one or both of these services. 

• The median length of JTPA enrollment for treatment group members who 
were enrolled in the program was 33 months; enrollees in the classroom 
training strategy had the longest enrollments (5.0 months) and those in 
OJT/JSA had the shortest enrollments (2.0 months). 
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For all enrollees in all service strategies, the median enrollment (i.e., the length of time 
at which half of enrollees had left the program) was 33 months. The classroom training 
service strategy had the longest median enrollment overall and for each target group, while 
OJT/JSA had the shortest enrollment There was considerable variation across the target 
groups in length of enrollment, with adult women and female youths having the longest 
enrollment, and adult men having the shortest enrollment This variation reflects the 
differences, discussed earlier, in the types of services they received. 

Findings on JTPA Service Receipt for Adults 

This section now turns to a discussion of the receipts of specific types of JTPA-funded 
services by the target groups in the study. Findings on JTPA service receipt are important for 
two reasons: they provide a needed context for interpreting the program impacts for each 
target group presented in the 18-month impact report, and they contribute to the ongoing 
discussion of service receipt differences across groups within JTPA by highlighting some 
possible reasons for them. 

The top panel of Table 4 presents JTPA service receipt rates for the adult target groups 
and key subgroups. The first row lists the service receipt rates for adult enrollees in each of 
five mutually exclusive service categories. The remainder of each panel lists selected subgroups 
of enrollees and the percentage of each subgroup who received each category of services. 

• Nearly 60 percent of the enrolled adults received some service designed to 
provide occupational training through either CT-OS or OJT. 

Table 4 shows that 38 percent of the adults received CT-OS and an additional 21 percent 
received OJT. Nineteen percent of the adults received job search assistance only, and the 
remainder received either basic education without training or other non-training services. 

• Adult enrollees with more employment barriers (i.e^ those who were high 
school dropouts, had limited work experience, and were receiving cash public 
assistance) were more likely to receive basic education without training and 
less likely to receive OJT or job search assistance as their only service than 
were those without these employment barriers. 

The three employment barriers listed above were associated with service receipt in 
different ways. Adults with limited work experience (who were therefore less attractive to 
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TABLE 4 



SERVICE RECEIPT RATES 
FOR ADULT AND OUT-OF-SCHOOL YOUTH ENROLLEES, 
BY SELECTED BASELINE CHARACTERISTICS 











Non- 




Misc. 










Training 


JSA 


Non-Training 


f 'harajitii trtai* 

vsuBTSClCrUUC 


Sample 


^Vp AC 

CT-OS 


ATT 

OJT 


Education 


Only 


Services 


and Subgroup 


Size 


(%> 


(%) 


(%) 


(%) 


(%) 


All Adult KnrnlW* 




Jo.U 


OA O 

20.8 


7.9 


19.2 


14.0 


Sex 














Male 


Z,Z50 


ZO.J 


/A.3 


7.2 


25.0 


17.2 


Female 


2,883 


47.2 


18.1 


8.4 


14.7 


11.6 


cioniciiy 














tw nuc t uuu nispuuc 


z,y4i 


30.6 


25.9 


9.1 


20.0 


14.5 


oiacj^ hod Hispanic 


1,4JZ 


50.5 


15.6 


2.7 


19.7 


11.5 


nispuuc 


coo 


38.7 


11.0 


15.8 


16.3 


18.2 


Other 


207 


55.1 


13.0 


3.9 


14.0 


14.0 


Barriers to Employment 














None 


1,828 


35.6 


24.1 


2.6 


23.5 


14.2 


1 


1,952 


37.5 


21.6 


6.7 


20.5 


n 7 


2 


1,092 


42.4 


16.8 


13.6 


13.2 


14.0 


3 


263 


40.3 


10.3 


28.5 


6.1 


14.8 


All Out-of-School 














Youth Enrollees 


2,147 


36.4 


15.5 


18.6 


10.9 


18.6 


Sex 
















959 


28.1 


18.0 


17.9 


13.2 


22.7 


Female 


1,188 


43.1 


13.4 


19.2 


9.0 


15.3 
















White tinn— W icnanir* 


1 , I JO 


oo o 


21.4 


i a a 
16.0 


13.1 


19.7 


quo nupuiic 


Cfifi 




1 A 

7.0 


31.8 


9.4 


18.9 


XUBMUil^ 


J/ J 


00 4 


1 A A 

10.9 


6.9 


7.2 


14.1 


Other 


46 


41.3 


15.2 


10.9 


6.5 


26.1 


Barriers to Employment 














None 


499 


39.7 


27.1 


3.0 


15.4 


14.8 


1 


798 


37.8 


16.4 


13.5 


12.9 


19.3 


2 


638 


33.5 


9.1 


30.4 


6.7 


20.2 


3 


207 


31.4 


3.4 


39.6 


5.3 


20.3- 



SOURCE: MDRC calculations from Background Information Form responses and from program 
enrollment and participation data from the 16 SDAs. 
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employers) and adults receiving cash public assistance (who therefore had an income source 
during JTPA participation) were more likely to receive classroom-based services such as CT- 
OS and basic education without training than adults without these barriers. They were also 
less likely to receive employment-based services such as OJT and job search assistance as a sole 
service. Adult high school dropouts were less likely to receive CT-OS (possibly because many 
training providers require a diploma or GED) and more likely to receive basic education 
without training than high school graduates or GED recipients. 

• Women tended to receive classroom-based services, while men tended to 
receive employment-based services. The employment barriers analyzed in this 
study account largely for the gender differences in OJT receipt but very little 
for the differences in CT-OS receipt There are several possible explanations 
for this remaining difference. 

Overall, 56 percent of the adult women — compared with 34 percent of the men - 
received either CT-OS or basic education without training. By contrast, 49 percent of the 
adult men - compared with 33 percent of the women — received OJT or job search assistance 
only. 

When adult men and women with similar employment barriers are compared (an analysis 
that is not shown in this table), most of the difference in receipt of OJT disappears. 
Apparently the gender differences in the receipt of these services - which are intended to 
place people in OJT - are largely related to job-readiness. However, the gender differences 
in receipt of CT-OS are not associated with differences in the employment barriers analyzed 
in this report. Other possible explanations include differences in client preferences, client 
characteristics or employment barriers, or local assessment and service referral practices. 

• Black and Hispanic adult enroilees were more likely to receive CT-OS and 
less likely to receive OJT than white enroilees. The employment barriers 
analyzed in this study accounted for some of the ethnic differences in service 
receipt, and there are several possible explanations for the remaining 
differences. 

Fifty-one percent of the black adult enroilees and 39 percent of the Hispanic adult 
enroilees received CT-OS compared with 31 percent of the white adult enroilees. By contrast, 
26 percent of the white enroilees received OJT compared with 16 percent of the black 
enroilees and 11 percent of the Hispanic enroilees. When individuals with similar employment 
barriers are compared, differences in service diminish, although less than was the case for 
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gender differences. Unlike the case for gender differences in service receipt, employment 
barriers do not account for a larger portion of service receipt difference in OJT and job search 
than in classroom training. As was the case for the gender analysis above, there are several 
possible reasons for these remaining ethnic differences. There may be differences in client 
preferences or other characteristics and barriers not analyzed here; employer hiring practices 
may make OJT and job search more difficult to provide for minorities; or local assessment and 
service referral practices, which may have been adopted in response to the reality of the labor 
market, may be different for whites and minorities. 

• The median length of JTPA enrollment for adults in the five service categories 
ranged from almost five months to only one month. 

Overall, the median length of enrollment for adult enrollees was 3.2 months. For those 
who received CT-OS, the median length of enrollment was longest at 4.8 months, and for those 
who received job search assistance only, it was shortest at one month. The median length of 
enrollment for adults who received OJT was 25 months. 

Findings on JTPA Service Receipt for Out-of-School Youths 

The bottom panel of Table 4 presents the service receipt findings for the out-of-school 
youth target groups and key subgroups. 

• Just over half of the out-of-school youth enrollees received some form of 
occupational training — either in the classroom or on the job — and a 
substantial percentage received bask education without training. 

Table 4 shows that 36 percent of the out-of-school youth enrollees received CT-OS and 
an additicnal 16 percent received OJT; i.e., youths were somewhat less likely than adults to 
receive these services. Nineteen percent of the out-of-school youths received basic education 
and 19 percent received miscellaneous services without training; i.e., youths were more likely 
than adults to receive these services. The remaining 11 percent received job search assistance 
only, making youths less likely than adults to receive job search assistance as their only service. 

• Youths with two or three employment barriers were more likely than those 
with fewer barriers to receive bask education without fining; they were 
less likely to receive CT-OS, OJT, or job search assistance as a sole service. 
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Thirty percent of the youths with two of the employment barriers analyzed (limited work 
experience, no high school diploma or GED, and/or cash public assistance receipt) and 40 
percent of youths with all three barriers received basic education without training compared 
with 3 percent of those with no employment barriers and 14 percent of those with one barrier. 
(These groups are shown in the lower panel of Table 4.) Forty-three percent of the youths 
with no employment barriers received the employment-based services of OJT or job search 
assistance only compared with 16 percent of those with two barriers and 9 percent of those 
with three barriers. The differences in CT-OS receipt rates were much smaller, but still 
significant 

• Female youths were more likely to receive CT-OS and less likely to receive 
OJT or job search assistance only than male youths. The employment 
barriers analyzed in this study accounted for some of the differences in 
service receipt among male and female youths, with several explanations 
possible for the remaining differences. 

Forty-three percent of the female youths received CT-OS compared with 28 percent of 
the male youths. Thirty-one percent of the male youths received OJT or job search assistance 
only compared with 22 percent of the female youths. Employment barriers accounted largely 
for the male/female differences in OJT and job search assistance only receipt rates, but only 
partly for the differences in CT-OS receipt rates. The same factors listed earlier for adults 
could also explain the remaining differences here. 

• White youths were more likely than black youths to receive OJT or job search 
assistance only and less likely to receive CT-OS or basic education. The 
employment barriers analyzed in this study accounted for some of the 
differences in service receipt between white and black youths, with several 
explanations possible for the remaining differences. 

Thirty-five percent of the white youths received OJT or job search assistance only 
compared with 16 percent of the black youths. By contrast, 65 percent of the black youths 
received CT-OS or basic education without training compared with 46 percent of the white 
youths. As was the case with adults, when youths with similar employment barriers are 
compared, some differences in service continue to appear, though employment barriers account 
for somewhat more of the service differences in employment-based services (OJT and job 
search assistance only). The possible explanations for service differences mentioned earlier for 
adults could also explain the remaining differences here. 
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• The median length of enrollment for youths in the five service categories 
ranged from over five months to less than one month. 

Overall, the median length of enrollment for out-of-school youths was 3.6 months, slightly 
longer than the median length of enrollment for adults. The median length of enrollment was 
S3 months for youths who received CT-OS, 3.1 months for those who received OJT, and .8 
months for those who received job search assistance only. 

Findings on Differences Among the Study Sites 

This section shifts from a focus on JTPA enrollment and services for the entire sample 
and for target groups to a discussion of differences among the 16 study sites. It begins by 
describing the variation among the sites and then discusses efforts to determine whether there 
are links between sites' patterns of enrollment and services and their administrative practices. 

• The 16 SDAs in the study varied widely in their service emphases. 

While diversity is expected, its magnitude is surprising. For example, the proportion of 
adult treatment group members recommended for the OJT/JSA service strategy ranged from 
7 percent to 79 percent, and the proportion of adult enrollees who actuaUy received OJT 
ranged from 4 percent to 54 percent Similarly, one SDA recommended the classroom training 
service strategy for just 6 percent of its adult treatment group members, while another 
recommended it for 77 percent of them. The variation in classroom training service receipt was 
nearly as great Large variations in recommended services and service receipt were also found 
for youths: for example, the rate at which youths were recommended for the OJT/JSA service 
strategy ranged from 1 percent to 71 percent, and the service receipt rate for OJT ranged from 
0 percent to 66 percent 

This wide variation may be due, at least in part, to local economic and other conditions, 
the availability of service providers, and the characteristics of the eligible population and of 
those recommended for services. It probably also reflects the decisions that SDAs made 
regarding recruitment, screening, and other administrative intake processes; the service providers 
they selected; and their response to JTPA's performance standards. This variation in services 
recommended and received provides an important context for interpreting findings presented 
in the 18-month impact report 
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• Sites differed in the proportion of the research sample who had two or more 
of the employment barriers analyzed (lack of a high school diploma or GED, 
limited work experience, and/or cash public assistance receipt) and a ose who 
had none of these barriers. 

For example, among adults, the proportion of the sample with two or more barriers 
ranged from 12 percent to 47 percent across the 16 sites, and among youth, this proportion 
ranged from 21 percent to 52 percent. It was not possible to determine whether this variation 
resulted from characteristics of the pool of eligible individuals in the different SDAs or labor 
market conditions — both factors over which SDAs have no control — or from the SDAs' 
levels of recruitment activities or selection/screening processes or from choices made by 
applicants. Therefore, the analysis is not able to attribute site differences in the overall 
research sample to differences in site practices. 

Cross-site comparisons of enrollment rates for groups within the research sample — 
defined by their number of barriers to employment — may be somewhat useful as an indication 
of targeting. Once people ar<> part of the research sample (i.e., they have applied and the 
SDA has found them eligible and appropriate for JTPA), SDA administrative practices may 
well be a key factor affecting the proportion of the treatment group that enrolls in JTPA. 

• Most sites in the study did not appear to establish clear enrollment targeting 
concerning the key employment barriers examined in the study. 

Keeping in mind the caveats noted above, it appears that only one SDA consistently 
focused on the more disadvantaged, in the sense that the SDA enrolled a much larger 
proportion of treatment group members who had two or three of the employment barriers 
analyzed than treatment group members who had none of them. A few SDAs did just the 
opposite, focusing on the more job-ready, meaning that they enrolled a higher percentage of 
those in the sample with none of the employment barriers analyzed. A few sites exhibited one 
pattern for adults and the opposite pattern for youths. However, most sites enrolled 
approximately equal proportions of both job-readiness groups. 

The final part of this summary examines whether there are links between site practices 
and enrollment patterns and service emphasis. Ibis is a much less rigorous type of analysis, 
more in the nature cf developing hypotheses for future research. This tentative stance is 
appropriate because of the small number of SDAs included in the study, and the limited 
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available information about SDAs' recruitment, screening, and assessment practices as well as 
the role of choices made by applicants* Furthermore, because the 16 SDAs differed in many 
ways, the analysis is only able to identify associations between characteristics of SDAs and 
enrollment patterns, rather than real causal relationships. Nevertheless, because of the strong 
policy interest in the influence of many local administrative decisions on JTPA recruiting and 
services, this analysis did explore several hypotheses about why SDAs might be more or less 
likely to focus on the more disadvantaged. 

• Based on limited evidence from the 16 study SDAs, there are indications that 
SDAs that did particularly well on key performance standards focused on 
more disadvantaged youths and enrolled more Job-ready adults* The different 
structures of adult and youth performance standards at the time the research 
sample was recruited for and enrolled in JTPA may explain this unusual 
finding* 

Among the 16 sites, SDAs that substantially exceeded their performance standards were 
more likely to focus enrollment on adults with none of the employment barriers analyzed th^n 
were SDAs that marginally exceeded their standards. Further, they also focused on the 
OJT/JSA service strategy. However, the opposite was the case for youths: SDAs that 
substantially exceeded their youth performance standards tended to focus enrollment on the 
more disadvantaged youths and to recommend a higher percentage of youths to the "other 
services 11 strategy. 

One possible explanation for these different findings for adults and youths may be that 
performance standards for youths included "positive termination'* outcomes such as completion 
of schooling or attainment of employment competencies. In contrast, for adults the 
performance standards in effect when the research sample was recruited to JTPA dealt 
exclusively with measures related to job placement This difference could have permitted, or 
encouraged, SDAs to enroll a more disadvantaged youth population - for some of whom job 
placement at termination from JTPA was not seen to be an appropriate goal - without 
jeopardizing their ability to meet performance standards. 

Again, it should be noted that this discussion must remain tentative because the 16 SDAs 
varied on many grounds, not just their performance measures; other factors could be causing 
the observed association between performance and enrollment practices. 

• Analyses failed to find clear associations between SDA practices and 
enrollment targeting in a number of jther areas* 
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The limited data from the 16 SDAs did not support several other hypotheses to account 
for differences in enrollment patterns and services emphasis. (The caveats noted above also 
apply to these findings.) For example, some argue that SDAs relying to a high degree on fixed 
unit price, performance-based contracts will serve more employable, less disadvantaged groups. 
Among the 16 study sites, these SDAs did not appear to focus more on enrolling the more 
job-ready than did SDAs that primarily used cost-reimbursement contracts. Similarly, SDAs 
that were operated by PICs were just as likely to focus on enrollment of the more job-ready 
as were SDAs operated by government 
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CHAPTER 1 
INTRODUCTION 



For nearly a decade, the Job Training Partnership Act of 1982 (JTPA) has stood as the 
nation's largest and most visible commitment to providing employment and job training 
opportunities for economically disadvantaged individuals. JTPA established a nationwide network 
of more than 600 largely federally funded, state-supervised, but locally operated partnerships 
between the public and private sectors to arrange for services intended to increase the 
employability of eligible adults and youths. Each year since its formal start-up in 1983, Title II- 
A (the main title of the legislation) has received close to $2 billion and enrolled about 1 million 
individuals - which represents less than 10 percent of the JTPA-eligible population. 

JTPA was established at a time when the concept of social program accountability gained 
a prominent place on the national public policy agenda. Building on its expectation that JTPA 
would be "an investment in human capital and not an expense," Congress established explicit goals 
for the new employment and training system: increases in employment and earnings and 
reductions in welfare dependency. For the first time, legislation called for a national system of 
outcomes-based performance standards, complete with rewards and penalties, to lead to the 
achievement of these goals. The legislation also required the U.S. Department of Labor (DOL), 
the federal agency that oversees JTPA, to conduct an evaluation to determine whether the goals 
were met 

In 1986, DOL initiated the National JTPA Study, a major effort to evaluate rigorously, in 
a limited number of the local service delivery areas (SDAs) 1 across the country, the impact that 
JTPA had on participants* employment, earnings, and welfare receipt. Following the 
recommendation of a technical advisory committee, DOL adopted and oversaw the successful 
implementation of a social experiment using a classical random-assignment design. Between 1987 
and 1989, more than 20,000 individuals in 16 local areas across the country were randomly 
assigned to a "treatment" group eligible to receive JTPA services or to a "control" group that was 
not eligible for JTPA-funded services but remained eligible for all non-JTPA-funded services in 
the community. The evaluation was designed to compare credibly and reliably how well the 

*In JTPA parlance, "service deliveiy area" refers to both the geographical area being served and the 
JTPA administrative unit serving it 
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treatment group fared compared with the control group in meeting the program's goals - thereby 
showing the difference, or 'impact,' that JTPA services made over and above what would have 
happened to these individuals without the program The study sought not only an estimate of 
JTPA's overall effectiveness in the participating SDAs, but also the impacts that JTPA's two major 
services - on-the-job training (OJT) and classroom training in occupational skills (CT-OS) - had 
on adult men, adult women, and out-of-school youths, as well as other important subgroups. 2 The 
study was a major and ambitious undertaking, one of the early examples of applying experimental 
techniques to evaluate an existing, ongoing delivery system. 

The research design for the resulting National JTPA Study has four main, interrelated parts. 
The first three focus on the 16 SDAs that participated in the evaluation: a study of their 
program implementation and operations, a study of the program impacts they achieved, and a 
study of benefits compared with c *ts. This report presents the findings for the first of these 
studies; a companion volume (Bloom et aL, 1993) presents the interim, 18-month impact findings. 
Thirty-month impact findings and the benefit-cost analysis will be published in the future. The 
fourth part of the research design involves an attempt to develop new nonexperimental methods 

to estimate program impacts. 

This report is intended to serve the National JTPA Study in three specific ways. First, it 
provides the programmatic context for understanding and interpreting the 18-month impact find- 
ings, as well as the future impact and benefit-cost findings. In meeting this purpose, it focuses 
primarily on analyzing, for approximately 17,000 sample members "pooled" across -the 16 SDAs, 
the major types of services that individuals were recommended for, the characteristics of these 
individuals, the services - if any - they actually received, and the duration of the services. 3 
Second, it examines the characteristics of the individual participating SDAs in order, insofar as 
possible, to present the variation across the SDAs in services and clientele, local environments, 
and organizational, management, programmatic, and administrative decisions. The final purpose 
demands both more speculation and greater caution: to test a number of widely held hypotheses 
about the possible relationships between SDAs' conditions and decisions, on the one hand, and 
the types of clients they enrolled and the services they provided, on the other. 

The results of the National JTPA Study are being published at a particularly important 



2The only major Title II-A group not included in the study is in-school youths. 

3-Services- in this report refers to employment, training, and other services directly related to 
employment; it does not correspond to the JTPA cost category called -services,' which refers primarily to 
transportation, child care, and other support services that assist people to attend the program. 
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juncture in the ongoing development of national employment and training policy. It is a time of 
increasing concern about the nation's ability to meet competitive economic challenges worldwide. 
The reported lack of appropriate basic literacy and jobs skills needed for a continually changing 
and demanding job market has led to increased attention on JTPA - to learn its strengths and 
weaknesses, to build on its successes where possible, and to make improvements in other areas. 
For this reason, the findings in this report, supporting those in the 18-month and future impact 
reports, should be particularly useful in providing evidence about JTPA's effectiveness and in- 
pointing to future directions. This introductory chapter provides a brief overview of the JTPA 
delivery system under study, a summary of the National JTPA Study, and an outline of the 
remaining parts of this report 

I. An Overview of the Title II-A JTPA System 4 

With strong bipartisan congressional support, JTPA was signed into law in October 1982. 
Title I established the state-supervised, locally operated system for delivery of the employment and 
training services to economically disadvantaged youths and adults authorized and funded under 
Title II-A. With funding at $1.8 billion, Title II-A immediately became by far the nation's largest 
federally funded employment and training program. 5 

The key provisions of JTPA were shaped in response to two main factors. First, a 
consensus emerged in the early 1980s that government employment and training programs needed 
to be more directly linked with the needs and expertise of the business community in order to 
meet the emerging economic challenges facing the nation. Second, there was a widespread view 
that the predecessor to JTPA, the Comprehensive Employment and Training Act (CETA), 
required major changes in the services that were authorized and in the responsibilities for 
oversight This view was coupled with inconclusive and sometimes contradictory findings from 
nonexperimental impact studies published during the debate on JTPA in the early 1980s. 6 The 



4 This overview describes the JTPA system as it existed during the period in which sites were selected, 
random assignment occurred, and services were provided to the treatment group. Amendments to JTPA, 
effective July 1993, will make several changes to the program, including creation of a separate youth title. 

*Thc Title II- A funding level has generally been sustained in nominal dollars, with only minor increases 
and cuts over the years. The Title II-A system is almost entirely federally funded, with the exception of a 
match requirement for the 8 percent state-level set-aside. For details, see Appendix A in Doolittle and 
Traecer, 1990. 

^Experimental impact studies entail random assignment of eligible individuals to a program group 
(referred to in this report as a "treatment group") or to a control group. The experiences of the control 
group then serve as the benchmark against which to compare the experiences of the treatment group. 
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3 



findings generally suggested that CETA training programs had very small, nonexistent, or negative 
impacts on the employment and earnings of men and modest impacts on women. 7 

These factors led to a significant shift in programmatic and administrative structures and 
relationships, and, overall, a substantial reduction of total federal resources earmarked for 
employment and training programs. In contrast to CETA, JTPA made substantial structural 
changes in four main areas: it assigned states a new administrative function, provided for a more 
substantial private sector role, focused resources on training activities while placing strict limits on 
other services and functions (including prohibiting public service employment and the payment of 
regular stipends to trainees), and established a system of outcomes-based performance standards 
intended to promote achievement of JTTA's job placement and welfare cost reduction goals. 
These four changes are the essential underpinning for the JTPA delivery system discussed in this 
report and evaluated in the National JTPA Study. 

A. The Role of Federal. State and Local Governments 

Under CETA, states were not substantively involved in the administrative oversight of 
training programs. Nearly all funding went directly from the federal government to local 
governments; DOL, through its 10 regional offices, supervised the system. However, JTPA gave 
states a substantial administrative role as the recipients of all Title II-A training funds - thereby 
eliminating the direct federal-to-local link. But states 9 control of funds was strictly circumscribed: 
they had to allocate a sizable proportion — 78 percent — of the funds to local SDAs, using a 
federal formula. The remaining 22 percent of the Title II-A funds were divided among a number 
of categorical set-asides over which states could exert some control. It is the use of the 78 
percent funds that is the subject of the National JTPA Study. 8 Further, the law explicitly stated 
that SDAs had sole authority over the use of the 78 percent funds (who to serve and what 
services to provide) as long as the uses were consistent with the act Indeed, the states' authority 



"(...continued) 

Nonexperimental impact studies of CETA typically established a "matched comparison group" (thought to 
be similar to the treatment group) rather than a control group created through random assignment. 

7 Sce, e.g., Bassi, 1983; Bloom and McLaughlin, 1982; Westat, 1981. For a summary, see Barnow, 1987. 

8 The set-asides were 3 percent for older Americans, 6 percent for incentives and technical assistance 
to local SDAs, 8 percent for the state education agency, and 5 percent for administration. The statute gave 
local SDAs some level of control over the 8 percent set-aside. 

In some study sites, services funded by portions of the 8 percent, 3 percent, and 6 percent set- 
asides at the state level were included in the National JTPA Study if the SDAs received and administered 
these funds. But the vast majority of services studied were funded through the 78 percent Title II-A 
category. 
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to review and approve local plans was also sharply limited by the act, giving governors little 
discretion beyond the criteria specified in the federal statute. This is the major reason that the 
National JTPA Study focuses entirely on local SDA-level services and programs* 

B. The Role of the Private and Public Sectors 

Building on a small program operated from 1979 to 1982 under Title VII of CETA, the 
JTPA delivery system was envisioned as consisting of a partnership — established in all local areas 
across the country - between government and business. Specifically, these partnerships were to 
be set up in "service delivery areas" (SDAs) that, generally, met a population threshold of at least 
200,000. In each area, local elected officials appointed members of nev/ly created private industry 
councils (PICs), which were required to have a majority of members representing the private 
sector and to be chaired by a private sector representative. PICs were authorized to provide 
policy guidance, approve local plans, and monitor the program. 

However, beyond these requirements and authorizations, the specific nature of the local 
government/business partnerships was designed to be determined through negotiations at the local 
level. Indeed, as long as the government officials and PICs agreed in writing to the local plan 
and the program administrative structure, the terms of the local partnership, such as the 
administrative and management roles that the government or PIC would assume, were, 
importantly, left to local discretion. Specifically, negotiations would determine which entit^ - the 
government, the PIC, or some combination of the two - wo\xld serve as the grant recipient and 
administrator. It was widely understood that this would likely result in differences in the relative 
strength of the PIC or elected officials in the various forms of partnerships across the country. 

C Allowed Use of Funds 

The third major area of change concerned the legal ways in which JTPA funds could be 
used and the balance in the use of funds among costs associated with training, support services, 
administration, and services to youths. As noted above, public service employment, CETA's most 
prevalent activity, was prohibited. A strict, 15 percent limit was placed on administrative costs, 
and the combination of administrative and support services costs (such as child care, 
transportation, and counseling) could not exceed 30 percent. Support services could include 
"needs-based payments" to participants, but not regular, hourly minimum-wage stipends that had 
been paid under CETA to provide people enrolled in training with a level of income support. 
Adult work experience was limited to six months, and half of the wages for youth and adult work 



experience had to count as support services. 9 A minimum of 70 percent of the funds had to be 
spent on training and, although there was no separate year-round youth program, 40 percent of 
the total funds had to be expended on youths. Further, two groups were specifically earmarked 
to receive services at a level proportionate to their incidence in the eligible population: high 
school dropouts and those welfare recipients who were required to participate in the Work 
Incentive (WIN) and successor programs. 

One of the most important features related to the use of funds was the authorization to 
use fixed unit price, performance-based contracts, in which full payment could not be made to a 
contractor until a participant was placed in a job. Costs for such contracts could be charged 
entirely to training. In sharp contrast to the more typical cost-reimbursement form of contracting, 
fixed unit price, performance-based contracts enabled SDAs to retain more of the limited 
administrative funds for central oversight and management rather than allocate them to 
contractors. But, as discussed in Chapter 2, this form of contracting was reported to give 
contractors an incentive to enroll participants who were judged more likely to succeed in training 
and employment 

D. Performance Standards and Reporting Requirements 

Building on a pilot effort to develop CETA performance standards, which DOL began in 
1979, early in the development of JTPA agreement was reached that it should be an outcomes- 
driven system with clearly stated goals and performance measures directly related to those goals. 
This thrust was viewed as the public sector counterpart to the private sector's emphasis on 
"bottom-line" profits, consistent with the theme of JTPA as an "investment." DOL was given 
broad authority to develop national standards; to provide a fair mechanism to permit states to vary 
local SDA standards to take into account different economic conditions, demographics, and 
services provided; and to establish consistent national reporting systems for performance 
measurement and general accountability. Six percent of the Title II-A funds were set aside, at 
the state level, for states to provide either performance incentive awards to SDAs exceeding their 
standards or technical assistance to underperformers. 10 States were also required to reorganize 
SDAs that failed to meet standards for two consecutive years. 

In implementing the performance standards requirements, DOL selected four outcomes- 



*There are some exceptions permitted, but few SDAs have taken advantage of them. 
10 The funds could also be used to provide services to the especially hard-to-serve. 



based standards for adult enrollccs and three for youth enrollees, 11 and provided states with a 
regression-based methodology to adjust local SDA standards, which most states adopted (This 
is the meaning of the term "adjusted performance standard" used later in this report) DOL 
further established reporting criteria and definitions, but did not provide a precise definition of 
the point at which "enrollment" or "termination" was to occur, which was then left largely to the 
discretion of individual SDAs. 12 

Together, these four major elements in JTPA - the new state role, the new role for the 
private sector, the limits on uses of funds (e.g., elimination of allowances to program participants 
and authority to use fixed unit price, performance-based contracting), and the emphasis on 
performance standards — along with an overall reduction in funds for the federal employment and 
training system - provided the basic framework for the nation's new employment and training 
system, a portion of which is the subject of the National JTPA Study. 

II. An Overview of the National JTPA Study 
A. Background 

Attempts to design evaluations capable of measuring accurately and reliably the difference 
that social programs make, over and above what would have occurred without the program, have 
increasingly drawn the attention of federal, state, and local policymakers, as well as program 
administrators and operators. Tie rapid growth in demands for accountability and the need for 
programs to show credible evidence of their value - particularly their impacts and cost- 
effectiveness — that has accompanied increased competition for limited government resources have 
also pushed government agencies and professional evaluators to adopt research methods that yield 
reliable assessments of program impacts and cost-effectiveness. 

In the judgment of many experts, an ideal impact evaluation for employment programs such 
as JTPA would consist of a comparison between the employment and earnings of those who were 
assigned to receive the program's services - the "treatment" or "experimental" group - and an 



n The four adult measures were: (1) entered employment rate, (2) entered employment rate for welfare 
recipients ("welfare entered employment rate"), (3) average wage at placement, and (4) cost per entered 
employment. The three youth measures were: (1) entered employment rate, (2) positive termination rate, 
and (3) cost per positive termination. 

i2The absence of consistent definitions for these two terms - and therefore a consistent identification 
of when an individual "counts" for the purposes of performance standards - had significant implications for 
the National JTPA Study, which are discussed in Chapter 3 of this report 
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identical group of people who were not able to receive the services, i.e., a control group. The 
comparison would reveal the difference (or impact) that the program made — over what would 
have occurred in the absence of the program — and, in doing so, the results for which the 
program could legitimately claim full credit 

This was, in fact, the advice of two national panels of experts convened in the early and 
mid 1980s, 13 which found that the impact evaluations conducted of JTPA's predecessors were - 
with rare exceptions — unable to meet the essential test of accuracy and credibility. The primary 
reason was the overwhelming difficulty posed in selecting individuals to form comparison groups 14 
and the lack of confidence that the comparison and treatment groups were as equivalent as 
possible in all respects, particularly regarding unmeasured characteristics such as motivation. One 
of these panels — convened by the National Academy of Sciences — concluded that the only 
alternative for the development of knowledge about program impacts was the use of random 
assignment to establish the two groups: "Our review of the research . . . shows dramatically that 
control groups created by random assignment yield research findings about employment and 
training programs that are far less biased than results based on any other method** (Betsey, 19f*f , 
p. 18). This is because random assignment, if done properly, assures that there will be .<< 
systematic differences in the two groups and therefore permits credible estimates of the effects 
of the program. Reaching the same conclusion, DOL's JTLS Research Advisory Panel explicitly 
recommended that the evaluation of JTPA utilize random-assignment experiments - accompanied 
by further exploration of the potential of nonexperimental methods to provide reliable estimates 
of program impacts. 

As a result, in early 1986 DOL issued a Request for Proposals (RFP) to conduct an 
evaluation of programs operated by the local SDAs set up under Title I and funded under Title 
II-A of JTPA As described in the first implementation report (Doolittle and Traeger, 1990), the 
initial research design pro r Dsed by DOL and the final research design implemented by the two 
contractors that DOL selected were fairly consistent except in one critical respect — the method 
of selecting the sites to be included in the study. In summary, the initial plan envisioned a 



13 Thc first, the Committee on Youth Employment Programs, was convened in 1983, at the request of 
DOL, by the National Research Council of the National Academy of Sciences to assess knowledge about 
youth employment and training programs. See Betsey, 1985. The second, the Job Training Longitudinal 
Survey (JTLS) Research Advisory Panel, was convened by DOL to provide advice on the plan to evaluate 
JTPA See JTLS Research Advisory Panel, 198S. 

14 *Our review of research that used such constructed comparison groups revealed . . . basic problems 
that repeatedly jeopardized the validity of the inferences to be drawn.* See Betsey, 198S, p. 17. 
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random selection of up to 20 SDAs for the study; under such a plan, it could have been argued 
that the impact findings for the 20 SDAs were generalizable to the JTPA system of more than 
600 SDAs. However, by an early point in the site recruitment process, a sizable proportion of 
the randomly selected SDAs had declined, for a number of reasons, to participate, and this 
approach was discontinued. (See Doolittle and Traeger, 1990.) Further site recruitment led to 
the selection of 16 diverse SDAs that were interested in and able to participate in the study. 
Hence, the findings from the various components of the National JTPA Study are explicitly not 
generalizable to the JTPA system. However, comparisons in this and the Doolittle and Traeger 
(1990) report generally show that the 16 SDAs reflect the overall diversity of the 600-plus SDAs 
in the people they served and the services they provided, and are similar to the larger JTPA 
system across several key dimensions, such as unemployment and poverty rates. (See Doolittle 
and Traeger, 1990.) Importantly, these 16 SDAs, although volunteers, ranged from those that 
substantially exceeded their performance standards to those that failed to meet them — much like 
the larger JTPA system. In short, although these SDAs are not statistically representative of the 
JTPA system, there is no evidence that they include only the highest or the lowest performing 
local programs; rather, across a number of key dimensions, they appear to reflect the considerable 
diversity in the system. 

B* Components of the National JTPA Study 

As noted earlier, the National JTPA Study consists of four major components. The first, 
a study of the implementation and operations of the programs in the 16 participating SDAs, is 
the subject of this report The second - a study of program impacts - is closely linked to the 
third, an analysis of program benefits compared to costs. The final component is an attempt to 
develop nonexperimental methods to estimate program impacts. 

The impact and benefit-cost studies are based upon an experimental design, using random 
assignment to treatment and control groups, called for by DOL in the RFP for the evaluation. 
DOL identified three major objectives for the impact study, which are discussed in more detail 
in Chapters 3 and 4. First, the random-assignment process should interfere as little as possible 
with the normal operations of the SDAs. Second, the evaluation should produce impact estimates 
for JTPA services overall and for four major target groups — adult men, adult women, male out- 
of-school youths, and female out-of-school youths - in addition to a variety of subgroups (e.g., 



as defined by prior work experience, welfare receipt, and ethnicity). 

Third, as part of the impact study, DOL further sought, for each of the four major target 
groups, separate impact estimates for JTPA's two major services - on-the-job training (OJT) and 
classroom training in occupational skills (CT-OS) - on those who were recommended by local staff 
to receive the service. Hence, the goal here was to measure the degree to which each was 
effective as a service. (In the research findings presented here, OJT was merged into an OJT/job 
search assistance [JSA] service strategy because JSA often became the service for those for whom 
OJTs were not arranged.) To accomplish this without changing significantly or disrupting the 
SDAs* existing operations, a process - by which SDA staff designated a "recommended service 
strategy" for applicants - was successfully built into the SDAs* normal intake and assessment 
activities. This process consisted of their identifying a "pool" of individuals whom they had 
assessed as appropriate for one of the three recommended service strategies; 16 these individuals 

17 

were then randomly assigned to the treatment or control group. 

The first of these recommended service strategies aimed at immediate employment, either 
subsidized or unsubsidized; the "anchor" services for this strategy were therefore OJT and, since 
it was so often used in combination with OJT, job search assistance. This strategy is therefore 
referred to as the OJT/JSA service strategy in this report. In contrast, the second strategy was for 
people for whom the SDA recommended the development, and acquisition of occupational skills 
before job-seeking activities, and was therefore "anchored" in CT-OS. 18 It is referred to as the 
classroom training (CT) service strategy in this report Finally, a third strategy - referred to in this 
report as the other services service strategy - was recommended for individuals for whom neither 
of JTPA's two main service strategies was considered appropriate, e.g., for basic education 



15 "Ettanicity* in this report includes "race." Original plans called for designating minority and non- 
minority youths as separate target groups and treating male and female youths as a subgroup. Following 
the recommendation of the Technical Advisory Panel for Part B, the impact analysts shifted to treating male 
and female youths as target groups, and whites, blacks, and Hispanics as subgroups. The Part A analysis 
followed this shift in terminology. 

16 In a number of SDAs, creating a large enough pool to allow for the establishment of a control group 
required increased recruitment efforts. See Doolittle and Traeger, 1990. 

"Chapters 3 and 4 discuss this in detail. 

18 Even though services other than the "anchor' services were permitted in each of these two strategies 
- eg basic education could be provided as part of either strategy - it was intended that clients would 
receive at some point, the strategy's "anchor" service(s). However, in order to make possible independent 
impact 'estimates of the effects of the OJT/JSA and classroom training strategies, neither of these two main 
strategies was designed to permit the other - eg., OJT could not be a recommended activity for the people 
in classroom training strategy, and vice versa. It was also recognized that not all clients recommended for 
a strategy would actually receive it See Chapter 3 for a detailed description of these service strategies. 
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unlinked to the OJT/JSA or classroom training service strategy. 

SDAs reported that the process of assigning individuals to these treatment strategies went 
smoothly and generally caused minimal disruption to their normal operations, and the evidence 
presented in this report shows that, by and large, the majority of individuals received the "anchor" 
service(s) for the strategy to which they were assigned. Hence, while the research design for the 
impact and benefit-cost components had some limitations, it enabled random assignment to be 
implemented in an ongoing, mature program with a minimal level of disruption to normal 
operations; and it enables the researchers to answer DOL's key questions about JTPA's 
accomplishments in the study sites: the impacts for target groups and key subgroups, overall and 
by activity. 19 

HI. The Structure of the Study and the Content of Its Reports 

DOL divided the original RFP into two parts. The Part A contractor was to be responsible 
for site selection, implementation and monitoring of random assignment and sample intake 
procedures (to ensure the integrity of the experimental design), and documenting, to a limited 
degree, the local environment, delivery structure, and nature of services provided by the sites. 
MDRC, with Abt Associates Inc. as a subcontractor, was awarded Part A. The Part B contractor 
was to be responsible for the impact studies, the benefit-cost analyses, and the nonexperimentai 
study, as well as most of the data collection and survey work. Abt Associates Inc., with ICF, 
MDRC, New York University, and National Opinion Research Center (NORC) as subcontractors, 
won Part B. 

A number of reports will have been generated by the National JTPA Study. First was a 
research design report (Bloom et al., 1990). It was followed by an implementation report 
(Doolittle and Traeger, 1990), which documented site selection, the nature and reasons for 
changes in the research design as more was learned about site operations, and start-up of the 
study at the 16 SDAs. This was the first of two Part A reports addressing the first component 
of the National JTPA Study - the analysis of program implementation and operations. The 



19 It is important to note that the research was not designed to answer whether OJT/JSA or classroom 
training would be a more effective strategy for the same people. This is because, to address this 
experimentally, individuals would have to have been randomly assigned to OJT/JSA or to classroom training. 
Rather, in part to meet the objective of causing minimal interruption in JTPA and measuring the program 
as it existed, individuals were first assessed as appropriate for a strategy and then randomly assigned. See 
further details in Chapters 3 and 4. 
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present report is the second and final one on this component of the study. A further report 
(Bloom, 1991) analyzed the characteristics of the research sample at the point of random 
assignment, and the services for which sample members had been recommended That report was 
based on data from the Background Information Form (BIF), which was completed on all sample 
members just pria to random assignment, and compared characteristics of the sample with data 
collected as part oi the Job Training Quarterly Survey (JTQS). 

A separate report (Bloom et aL, 1993) presents the 18-month impact findings, based 
primarily on information reported by respondents to a survey. The next impact report, covering 
30-month impact and benefit-cost findings, will draw on both survey data and a variety of 
administrative records (e.g., wage records from the Unemployment Insurance system, welfare 
payment records from state or local welfare agencies) and cost data provided by the participating 
SDAs. The final report will present the findings from the nonexperimental study. 

IV. The Purpose and Organization of This Report 

As described above, this is the second of two reports that are part of the first component 
of the National JTPA Study - the study of program implementation and operations. The main 
data sources for this report are three: the baseline data collected on the members of the 18- 
month study sample just prior to random assignment; the JTPA participant enrollment, 
participation, and termination information routinely collected on enrollees by the participating 
SDAs; summary data on SDAs from the JTPA Annual Status Report (JASR); and a limited 
amount of field research the DOL called for in the original RFP (see below). A special survey, 
described in Chapter 3, was also conducted. 

This report has three major purposes. First, in its broadest sense, the report provides the 
operational context for interpreting the 18-month impact findings (Bloom et al., 1993) as well as 
those to be published in the final impact report. "Context" in this framework includes quantitative 
information about the types of services for which JTPA applicants were recommended, their 
enrollment rates, and the types of services they received; information about the external factors 
affecting the participating SDAs, such as labor market conditions and the characteristics of the 
JTPA-eligible population; and qualitative information about the SDAs' areas of administrative, 
management, and organizational discretion — and constraint. 

In the evaluation RFP, DOL specified that limited resources be devoted to collecting 
qualitative data on the participating SDAs and the services they funded. Field research was 

12 



undertaken only as part of random-assignment-monitoring visits. Qualitative data were not 
systematically collected on the nature of the services provided, and only basic information about 
the "external factors' 1 was collected on the participating SDAs, such as unemployment data, wage 
level, and population data. 

The second major purpose of the report is to describe the flow of JTPA applicants from 
the point of application to JTPA to the point of enrollment (or nonenrollment). This is 
important to understand because it describes the process by which individuals were moved through 
the SDAs' intake procedures that would lead, ultimately, to their receiving or not receiving 
services. From both a research and programmatic perspective, this complicated process has critical 
relevance to recognizing the need to have the two sets of impact estimates that are found in the 
18-month impact report - one for those recommended for services, and one for those officially 
enrolled in JTPA by the participating SDAs. Hence, explaining this process and its relevance to 
understanding the impact reports is, again, a major, second purpose of this report 

The third purpose of the report is to explore the relationships between the types of services 
SDAs provided, the types of participants they served, and a variety of other factors. Some of 
these factors, such as the unemployment rate, were beyond the SDAs' control; others, such as the 
role of the government entity or the PIC, permitted some level of SDA discretion within statutory 
parameters. Given the limited number of SDAs in the study, however, such analyses must be 
viewed as quite speculative and exploratory. 

To meet these three purposes, the remainder of this report is divided into six chapters: 

• Chapter 2 provides the background information on the study sites and the 
sample for this and the 18-month impact report It describes the external 
environment in which the participating SDAs operated, such as labor market 
conditions, and the characteristics of the program participants the SDAs 
served during the years the study sample was selected. It also presents the 
key management, organizational, and administrative decisions that SDAs made 
in areas over which they had some degree of discretion. 

• Chapter 3 describes the flow of potential JTPA participants through the 
stages of the program, from application to enrollment. Together with Chapter 
4, it provides the essential context for understanding the subsequent chapters 
and the 18-month impact report 

• Chapter 4 discusses the key policy questions in the impact research and 
explains the research design choices made to address them. 

• Chapters 5 and 6 present the quantitative participation findings on, 
respectively, adults and out-of-school youths who were officially enrolled in 
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JTPA during the 18-month follow-up period for this report Each chapter 
starts by analyzing the JTPA services received and then presents differences 
by gender, ethnicity, and other subgroups. These chapters also present 
findings on how long enrollees stayed in the program. 

Chapter 7 offers an exploratory examination to determine if the kinds of 
services SDAs provided or the types of individuals they enrolled are related 
to their local conditions and/or their decisions about organization, 
management, and administration. 
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CHAPTER 2 
THE STUDY SITES AND THE SAMPLE 



This chapter provides background on the 16 local JTPA programs and the sample of 
individuals studied in this report. It discusses the local population and economic conditions of 
service delivery areas (SDAs) in the study, describes key aspects of their administrative 
structure, summarizes the characteristics of all program participants they served under Title II- 
A of JTPA over the three years in which the study sample was selected, and discusses the 
degree to which they succeeded in meeting performance standards. The chapter concludes by 
reviewing the characteristics of the sample analyzed in this report and in the 18-month impact 
report (Bloom et aL, 1993). When appropriate data are available, the chapter compares the 
study sites with all SDAs in the national JTPA system. 

The basic conclusion is that the sites and the individuals whose experiences are examined 
in this report resemble in many ways the JTPA program and its participants nationally and 
include much of its diversity. The 16 sites include several with very strong economies during 
the late 1980s, others experiencing modest growth, and still others recovering slowly from job 
losses in the recession of the early 1980s. Following national JTPA guidelines, the study sites 
made different decisions on key aspects of local administrative discretion, including the role of 
the private industry councils (PICs), their choice of service providers, and the way they 
structured contracts for services. Furthermore, their performance, measured in terms of key 
Department of Labor (DOL) standards and based on the experience of those leaving the 
program during the study years, showed similar diversity, with both strong and weaker 
performers included in the study. The members of the study sample are themselves a diverse 
group with many similarities to those sewed nationally. Their diversity allows analysis of 
program implementation and impacts based on demographic characteristics, welfare receipt, and 
employment history. 1 

Before presenting these findings on the sites and individuals in the study, it is useful to 



x See Bloom, 1991, for a detailed discussion of the characteristics of the sample and a comparison 
with characteristics of people served nationally by JTPA 
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provide some background on the selection of sites analyzed in this report The sites in the 
study were recruited to participate in late 1986 and 1987. An earlier report on this project 
discussed the many issues that arose during site selection and how the original goal of randomly 
selecting study sites from all local JTPA programs shifted to recruiting a diverse group of sites 
interested in and able to participate in the study. 2 The 16 study sites began random assignment 
between November 1987 and October 1988, and all sites completed this phase of the project 
by the end of September 1989. 

The definition of the sar pie analyzed in this report and the 18-month impact report 
(Bloom et aL, 1993) needs explanation. In the 16 study sites, a total of 20,601 individuals were 
randomly assigned to either the treatment group, which was given access to JTPA services, or 
a control group, which was not 3 Follow-up surveys were then administered Choices made 
to conserve project resources devoted to the survey led the research team to use a somewhat 
smaller sample for this report and the 18-month impact report (which relies on the first follow- 
up survey). To compress the time during which survey staff conducted the follow-up interviews, 
individuals were contacted at slightly different times relative to when they were randomly 
assigned. 4 Some individuals were surveyed as early as 13 months after the date they were 
randomly assigned, while others were surveyed as late as 22 months following random 
assignment 5 

The methods available for analyzing variable-length follow-up are quite complicated. 6 
Therefore, to simplify the analysis of program implementation and impacts, the research team 
selected a sample of individuals for whom survey contact was scheduled at least 18 months after 
their date of random assignment 7 This sample, called the "18-month study sample" in this 



2 Doolittle and Traeger, 1990. 

3 In Bloom et aL, 1993, these 20,601 individuals are referred to as the "experimental sample." 

4 Random assignment began in the first sites in November 1987 and ended in the last site 23 
months later. If all persons had been contacted on the 18-month anniversary of their random 
assignment, these contacts would have been spread over a 23-month period. 

5 Most of the sample was interviewed between IS and 20 months following random assignment. 

Problems of analysis occur because the sample changes as the length of follow-up increases. For 
this survey, as noted above, follow-up varied in length from 13 to 22 months. Thus, some of the 
individuals analyzed in data collected 13 months after random assignment would not be in the sample 
as follow-up lengthened. Even with complicated statistical modeling, it would be difficult to determine 
if changes in impacts over time occurred because of time trends in program impacts or differences in 
the composition of the sample. 

7 Eighty-four percent of these people actually completed the first follow-up survey. 
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report and the 18-month impact report, consists of 17,026 individuals assigned to either the 
treatment group or the control group before July 1989. Much of the analysis in this report 
concerns the experiences of the 11,474 individuals in this sample who were assigned to the 
treatment group and who, by definition, had access to JTPA services; these individuals make 
up the "18-month treatment group sample." Tne final impact report, based on longer follow- 
up, will rely on both a second follow-up survey and administrative records. It will analyze the 
experiences of a larger sample. 

With this as background, this chapter now describes the characteristics of the study sites 
and compares them with what is known about JTPA nationally. It then provides a brief 
overview of the characteristics of the sample analyzed in this report 

I. The 16 Study Sites 

The sites in the study were recruited from among SDAs in the 48 contiguous states with 
at least 500 persons leaving programs funded under Title II-A programs ("tcrminees" in the 
language of program reporting) in program year 1984.** Table 2.1 lists the formal SDA name, 
a more descriptive, abbreviated site name used in this report, and each site's census region. 
It also lists the largest city within each site and the size of the 18-month study sample. Figure 
2.1 shows the approximate location of each site. This section reviews three features of these 
local sites that show considerable variation: the characteristics of the local population and 
economy, the administrative structure of the SDAs, and the characteristics of terminees served 
by the program during the period of random assignment 

A. Characteristics of the Local Population and Economy 

The 16 study sites are spread throughout the nation, with two in the Northeast, four in 
the South, seven in the Midwest, and three in the West Oakland and Jersey City are located 
in large metropolitan areas with substantial black, Hispanic, and other minority residents, but 



8 Program year 1984 (July 1984 through June 1985) was the latest year for which uata were available 
at the time site selection for the study began. People can leave the program vit\ positive (e.g., 
employment) or negative (e.g., failure to meet program requirements) outcomes. 
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FIGURE 2.1 

LOCATION OF THE NATIONAL JTPA STUDY SITES 
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no large central cities are included among the sites. 9 Among the sample are predominantly 
rural or small-town sites and mixed urban-suburban-rural sites. The largest cities within each 
of the 16 study SDAs range in population from Oakland (372,000 in 1990) and Omaha 
(336,000) to Thief River Falls (under 10,000). 

Table 22 provides more evidence of this variety. Three SDAs made up solely of cities 
stand out in population density (shown in the left column of the table): Jersey City, Oakland, 
and Providence. Butte, Larimer County, Northwest Minnesota, and - to a lesser degree - 
Cedar Rapids and Springfield fall at the other extreme and are largely rural in character. Fort 
Wayne's relatively low population density is an average over eight counties that are 
predominately rural in character. The average for the ~ ample is above that for the nation as 
a whole, partly because rural SDAs with a small number of participants were not recruited to 
participate in the study. 10 

The poverty rate, presented in the second column of Table 22, shows similar variety. 
Sites located in large, heavily minority metropolitan areas (Jersey City, Oakland) have the 
highest poverty rates, but other urban sites with minority populations such as Corpus Christi 
(Hispanic) and Jackson (black) also have higher-than-average rates. Four other sites (Coosa 
Valley, Georgia; Heartland, Florida; Northwest Minnesota; and Springfield, Missouri) also have 
poverty rates slightly above the sample and national average. 

Economic characteristics at the time of random assignment summarized in the remaining 
columns of Table 22 reflect differences in regional economic conditions and the local economic 
base. As is the case nationally, the average unemployment rate among the sites masks great 
differences. Corpus Christi's residents experienced persistently high unemployment during the 
late 1980s, as the oil industry suffered an extended slump. At the other extreme, Providence's 
low unemployment rate resulted from New England's high technology boom of the same 
period, while the low rates in Cedar Rapids, Fort Wayne, and Omaha reflect the economic 
resurgence experienced by some middle-sized metropolitan areas in the Midwest. Decatur's 
high unemployment, however, illustrates that this recovery did not occur everywhere; in this 



^rge central cities often have many ageucies handling recruiting and program intake, which would 
have made implementation of random-assignment procedures very difficult. For example, the City of 
Los Angeles program involved more than 50 service providers at the time of site selection for the study. 

l0 The averages for the sample and for the nation are unweighted. Thus, all sites, regardless of the 
number of individuals served, are treated equally in calculating both averages. 
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manufacturing and food processing center, the recovery of the mid and late 1980s was very 
weak. The variation in SDAs' average earnings reflects higher average earnings in urban areas 
than in rural areas (e.g., Oakland versus Northwest Minnesota) and the importance of high- 
wage industries in some sites (e.g., petroleum in Corpus Chris ti an<* heavy manufacturing in 
Decatur). 11 Differences in the local economic base are further illustrated in column 5 of Table 
22y which presents the percentage of employees in the manufacturing, mining, and agriculture 
industries. The final column, on growth in retail and wholesale earnings, captures the effects 
of economic conditions in each SDA. Corpus Christi's economic downturn is starkly visible 
(minus 15.5 percent), as is the northeastern economic, boom of the late 1980s (Jersey City with 
9.9 percent and Providence with 9.7 percent). On all these measures, the sample average is 
similar to the national average. 

B. The Administrative Structure of the SDAs 

The passage of JIT A in October 19S2 established a new federal/state/local service delivery 
system to provide employment and training services to the economically disadvantaged. It also 
gave new areas of discretion — within certain requirements — to local SDAs in managing and 
administering the provision of services. Chapter 1 outlined the key changes in the nation's 
employment and training system brought about by the passage of JIT A; this section identifies 
three areas of discretion in which the responses of the 16 study SDAs varied substantially. 

1. The local JTPA service delivery system. The federal statute assigned responsibility 
for the delivery of services funded under Title II-A of JTPA to local level SDAs. Congress 
envisioned a "partnership" between government and business — specifically, between the elected 
officials of the local government and representatives of the private sector — and created a 
forum for this in the newly created PICs. But much was left to negotiations at the local level. 

Specifically, negotiations would determine which entity - the government, the PIC, or 
some combination of both — would play the following roles: 

* the grant recipient, which received and was held legally responsible for the 
JTPA funds allocated to the SDA by the state, and 

11 Average earnings *re calculated by dividing the total payroll reported to federal and state 
unemployment insurance programs by employers in the SDA by the number of employees in the SDA. 
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• the administrative entity, which administered the program, including 
developing the plan, selecting contractors, recruiting clients, and arranging 
services. 

If local negotiations determined that the PIC would play either or both of these roles, then the 
PIC would have to formally incorporate (in order to receive public funds) and hire staff to 
conduct the business of the PIC. Even in situations in which PICs did not perform either of 
these functions, they often established their own identity separate from the government by 
incorporating and/or hiring their own staff. 

The choices that the government and PICs made in the 16 SDAs in the JTPA study are 
shown in Table 23. 12 In six SDAs (labeled "Government-Operated"), the local government (or 
consortium of contiguous governments) acted as both grant recipient and administrative entity 
and the PIC chose not to incorporate or to have separate PIC staff. In another three SDAs 
(labeled TIC-Operated"), the PIC was an incorporated body with its own staff and served as 
administrative entity and grant recipient In the remaining seven, the government and PIC 
shared responsibilities and/or had various staffing arrangements. 

2. The selection and role of service providers . The federal statute also assigns to 
the SDA the responsibility for selecting and defining the roles of JTPA service providers, 
which can range from public agencies and community-based and other nonprofit organizations 
to private for-profit companies. 13 As Table 2.4 shows, during the period of random 
assignment, service providers for employment, training, and education services varied across the 
16 study SDAs. Perhaps surprisingly, the arrangement of OJTs - i.e., the placement of clients 
in OJT slots in subsidized training in either the private or public sector — involved private for- 
profit agencies in only 2 SDAs. More commonly, OJT was run directly by SDA staff in 11 
SDAs, with the state employment or job service — a traditional provider of OJT — also playing 
a role in 2 of these SDAs. Table 2.4 also shows that public education institutions - 
vocational-technical schools, community colleges, or universities — were providers of classroom 
training in 14 sites; and that proprietary schools were providers in 8 of the 16 SDAs. Further, 
job search assistance was provided by a range of agencies, but most typically by either the SDA 



12 The SDAs are grouped by their status at the suit of the study. The notes to the table indicate 
later changes. 

l3 Through its state coordination and special services plan, the state can influence this decision. 
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TABLE 2.3 



ORGANIZATIONAL STRUCTURE OF THE SDAs PARTICIPATING 
IN THE NATIONAL JTPA STUDY, BY SITE 

SDA Organizational Structure Funding Administrative Incorporated Separate PIC 
and Site Recipient Body PTC? Staff? 



Government-Operated 



Butte, Mont. 


Gov. Entity (a) Gov. Entity (a) 


No (a) 


XT /-A 

No (a) 


0«u4*r Hani He lows 


Gov. Entity 


Gov. Entity 


No 


No 


Decatur, 111. 


Gov. Entity 


Gov. Entity 


No 


No 


Heartland, Fla. 


Gov. Entity (b) Gov. Entity (b) 


No(b) 


No(b) 


Omaha, Neb. 


Gov. Entity 


Gov. Entity 


NO 


VT_ 

NO 


Springfield, Mo. 


Gov. Entity 


Gov. Entity 


No 


VI- 
NO 


PlC-Operated 


• 








Fort Wayne, Ind. 


PIC 


PIC 


Yes 


Yes 


Marion, Ohio 


PIC 


PIC 


Yes 


Yes 


Northwest Minnesota 


PIC 


PIC 


Yes 


Yes 


Government/Pi C-O Derated 










Coosa Valley, Ga. 


n/a(c) 


n/a(c) 


Yes 


No 


Corpus Christi, Tex. 


PIC 


Gov. Entity/ 


Yes 


Yes 






PIC (d) 






Jackson, Miss. 


Gov. Entity 


Gov. Entity 


Yes 


No 


Jersey City, N.J. 


Gov. Entity 


Gov. Entity (e) 


Yes 


Yes 


Larimer County, Colo. (0 


Gov. Entity 


Gov. Entity 


No 


No 


Oakland, Calif. 


Gov. Entity 


Gov. Entity/ 


Yes 


Yes 






PIC (g) 






Providence, R.I. (0 


Gov. Entity 


Gov. Entity 


No 


No 



SOURCE: Information collected by MDRC site representatives during the National JTPA Study. 

NOTES: (a) On July 1, 1990, the operator of this SDA changed from the Montana Department of Labor 
and Industry, a government entity, to Montana Job Training Partnership, Inc., a private nonprofit organization 
formed by the two PICs in Montana. At this time, the PIC became incorporated and developed a separate staff. 

(b) In December 1989, the PIC became the funding recipient and administrative body; in January 
1990, it became incorporated and developed a separate staff. 

(c) Both the funding recipient and the administrative body are the Coosa Valley Area Regional 
Development Center. This organization was established by state law with the consent of the PIC 

and the local government entity responsible for JTPA. 

(d) A government entity was originally responsible for administration; on July 1, 1988, 
the PIC began to function as both the funding recipient and the administrative body. 

(e) The Corporation for Employment and Training, Inc. (CET), was contracted by Jersey City 
to act as the administrative body. CET lost its contract with the city at the end of June 1990. 

(f) According to the criteria used in this table, this site should be included in the government- 
operated SDA category; however, MDRC site representatives included it in the goverrroent/PIC- 
operated SDA category because the PIC plays an especially significant role in deciding how to spend 
JTPA funds and choosing service providers. 

(g) The city of Oakland and the PIC shared this role. 
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or the job service, or both. Basic education, when it was offered, was provided by public 
education in 9 of 12 cases. But it was not offered in 4 SDAs, at least not as an identifiable 
stand-alone activity. 

The provision of recruitment and assessment was largely an SDA staff function, rather 
than being contracted out to other organizations (see Table 2S). SDAs were responsible for 
most or all recruitment activities in 10 of the 16 SDAs, while service providers were 
responsible for most or all recruitment in 4, and in 2 SDAs the function was shared equally. 
Most or all of the client screening and assessment were done by SDA staff in 9 sites, and by 
service provider staff in 4. In the remaining 3, both organizations played a role. 

3. Types of contracts. SDAs had a choice of structuring their legal relationship with 
contractors in one of two quite different ways: 

• cost-reimbursement contracts, in which the contractor was paid for services 
rendered regardless of the outcome for the enrollees, or 

• fixed unit price, performance-based contracts, in which full payment was, 
by law, contingent on enrollees 9 achieving specific outcomes such as job 
placement in a training-related job at a certain wage rate for a specified 
period of time. 

SDAs" choice of performance-based contracts has provoked considerable controversy. They can 
create incentives for the contractor to screen applicants carefully before accepting them, 
because of the risk of financial loss. (This risk was alleged to discourage service providers 
from accepting hard-to-serve clients.) There was, however, a major financial incentive for 
SDAs to adopt fixed unit price contracts: all costs of these contracts could be charged to the 
"training" cost category and none to administration, which had a ceiling of 15 percent of all 
local expenditures and was the source of funding for local planning, management, and other 
administrative tasks. In addition, SDAs that emphasized th" importance of surpassing their 
performance standards could "pass through" the obligation to meet or exceed them by 
structuring their performance-based service provider contracts to link payment to success in 
achieving or exceeding these benchmarks. With cost-reimbursement contracts, in contrast, the 
contractor's costs had to be charged to their proper cost category; some of the funds expended 
under these contracts were typically charged to "administration." During the 1980s, 
performance-based contracts became a common — and controversial — feature of JTPA, with 
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TABLE 2.5 



ROLES OF SDAs AND SERVICE PROVIDERS AT SELECTED POINTS 
IN THE CLIENT FLOW, BY SITE 



Who Is Responsible 

Who Is Responsible Who Is Responsible for Screening and 
for Recruitment? for Intake? Assessment? 



Site 


SDA 


Service 
Provider 


SDA 


Service 
Provider 


SDA 


Service 
Provider 


Butte, Mont, (a) 


None 


All 


None 


All 


None 


All 


Cedar Rapids, Iowa 


Most 


Some 


Most 


Some 


Most 


Some 


Coosa Valley, Ga. (b) 


Some 


Most 


None 


All 


None 


All 


Corpus Christi, Tex. 


Most 


Some 


Most 


Some 


Some 


Some 


Decatur, 111. 


All 


None 


All 


None 


All 


None 


Fort Wayne, Ind. 


AU 


None 


All 


None 


All 


None 


Heartland, Fla. 


Some 


Most 


All 


None 


Most 


Some 


Jackson, Miss. 


Some 


Some 


Some 


Some 


Some 


Some 


Jersey City, N J. 


Most 


Some 


Most 


Some 


Some 


Some 


Larimer County, Colo. 


All 


None 


All 


None 


Most 


Some 


Marion, Ohio 


All 


None 


All 


None 


Most 


Some 


Northwest Minnesota (c) 


Some 


Some 


None 


All 


None 


All 


Oakland, Calif. 


Some 


Most 


Some 


Most 


Some 


Most 


Omaha, Neb. 


Most 


Some 


Most 


Some 


Most 


Some 


Providence, R.I. 


Most 


Some 


Most 


Some 


Most 


Some 


Springfield, Mo. 


Most 


Some 


Most 


Some 


Most 


Some 



SOURCE: Information collected by MDRC site representatives during the National JTPA Study. 



NOTES: (a) In Montana, the state contracted with the Montana Job Service Division, a service 
provider, to operate the JTPA program. 

(b) In Coosa Valley, staff of Berry College, a service provider, are solely responsible 
for intake, screening, and initial assessment. Other service providers can perform further 
assessment. 

(c) Northwest Minnesota's primary service provider, the Minnesota Job Service, is solely 
responsible for intake and assessment under a contract with the PIC. 
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disputes over their use and allegations that some service providers received "excess revenue" 
or profits, leading to revisions of contracting rules by DOL, investigations of alleged abuses by 
DOL's Office of Inspector General, and legislative proposals for reforms. 14 

As Table 2.6 shows, the SDAs varied considerably in the percentage of training costs 
spent through performance-based contracts. Seven SDAs did not use performance-based 
contracts at all, and an eighth used them only slightly. At the other extreme, three SDAs 
(Coosa Valley, Jackson, and Northwest Minnesota) used peiformance-based contracts for more 
than 50 percent of both adult and youth training expenditures. 

C Characteristics of Program Tcrminces 

Tables 2.7 and 2.8 provide information on the characteristics of individuals leaving 
programs funded under Title H-A of JTPA ("program terminees"), by study site, during 
program years 1987 through 1989, which included the period of random assignment 15 On all 
these measures, the average for the study sites was similar to that for all SDAs. 16 The large 
differences in the ethnic distribution of terminees across the sites, shown in Table 2.7, primarily 
reflect differences in the local population. In general, SDAs from metropolitan areas have a 
much higher proportion of black and Hispanic (and in some cases Asian) terminees than is the 
case in rural areas. Corpus Christi (79 percent), Jackson (86 percent), Jersey City (95 
percent), and Oakland (93 percent) have the highest percentages of nonwhite and Hispanic 
terminees. 

Table 2.8 shows the proportion of terminees for each of seven obstacles to employment; 



14 For DOL's response in the late 1980s, see Federal Register, 1989. For an example of the 
Inspector General's criticisms, see U.S. Department of Labor, Inspector General, 1991. 

15 Terminees served between program years 1987 and 1989 will be a larger group than the 18- 
month treatment group sample analyzed in this report for several reasons. First, random assignment 
was not done for the entire three years in any site; random assignment began in the earliest site in 
November 1987 and ended in the last site in September 1989. Second, the study excludes in-school 
youths and the summary statistics on people served include them. Finally, there were some limited site- 
specific exclusions from random assignment when local programs faced severe recruitment difficulties 
in meeting enrollment targets for particular groups or program intake was decentralized to organizations 
serving few people. 

"Again, these are unweighted averages using SDA data from the JTPA Annual Status Report for 
both the study sites and the national program. This method of calculating averages was chosen because 
the SDA rather than the individual, is the focus of the analysis in this chapter and tor ease of 
presentation. An earlier report on the baseline Characteristics of the entire study sample (Bloom, 1991) 
examined similar issues using individual data and also found that the study sample closely resembles 
national program terminees. 
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TABLE 2.6 



LEVEL OF USE OF PERFORMANCE-BASED CONTRACTS 
AMONG SITES IN THE NATIONAL JTPA STUDY, 
FOR ADULTS AND YOUTHS 





Percentage of Adult 




Percentage of Youth 




Training Expenditures 




Training Expenditures 


Level of Use 


Resulting from Performance- 


Level of Use 


Resulting from Performance- 


end Site 


Based Contracts (a) 


and Site 


Based Contracts (a) 


Hifih Use rtrt 




High Use Art 




VcUUM v U1GV| VPS* 


n 


Jackson, Miss. 


80 


iNuruiwosi [Yi iniVcSOil 


01 


Coosa Valley, Ga. 


80 


Jickion Miu 


ou 


Northwest Minnesota 


68 






Oakland p»1if 

VaANUM| Will . 




Moderate Use to 




Corpus Christi, Tex. 


51 


Corpus Chris ti, Tex. 


46 


Moderate Use <c\ 




Omaha, Neb. 


38 




Oakland, Calif. 


37 


Omaha, Neb. 


28 


Providence, R.I. 


25 


Heartland, Fla. 


21 


Heartland, Fla. 


22 


Providence, R.I. 


18 


Little or No Use frft 




lavuc or no use tat 




Butte, Mont. 


2 


Butte, Mont. 


o 

V 


Cedar Rapids, Iowa 


0 


Cedar Rapids, Iowa 


0 


Decatur, 111. 


0 


Decatur, 111. 


o 


Fort Wa;yn^ Ind. 


0 


Fort Wayne, Ind. 


0 


Jersey City, N.J. 


0 


Jersey City, N.J. 


0 


Larimer County, Colo. 


0 


Larimer County, Colo. 


0 


Marion, Ohio 


0 


Marion, Ohio 


0 


Springfield, Mo. 


0 


Springfield, Mo. 


0 



SOURCE: MDRC calculations from program year 1988 fiscal records collected by Abt Associates Inc. 
from the 16 SDAs. 



NOTES: Under a JTPA performance-based contract, service providers can receive partial payments 
only when they attain performance benchmarks. Also, full payment under these contracts is 
contingent upon three conditions: completion of training, placement in a training-related 
job, and receipt of a specified wage. All payments made under these contracts can be 
charged to training expenditures. 

(a) In some SDAs, the total for training expenditures includes some JTPA "8 percent" education 
funds, which are spent in the SDA but are not under the direct control of SDA staff. This may affect the 
percentages listed, but does not change the composition of the level of use categories. 

(b) A site had "high use" of performance-based contracts if payments under this type of 
contract accounted for over SO percent of total Title II- A training expenditures. 

(c) A site bad "moderate use" of performance-based contracts if payments under this type of 
contract accounted for IS to 49 percent of total Title II-A training expenditures. 

(d) A site had "little or no use" of performance-based contracts if payments under this type 
of contract accounted for under IS percent of total Title II-A training expenditures. 
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in general, the average for the study sites was similar to that for the nation. Six of these seven 
obstacles affected, on average, 25 percent or fewer of both study site and national program 
terminecs, the exception being "unemployed for IS weeks or more during the past 26 weeks." 
Sites varied widely in the proportion of terminecs for each measure. For example, Coosa 
Valley, Corpus Christi, and Providence had tenninees with the greatest educational problems, 
based on their high percentages of school dropouts and low-level readers; Butte and Cedar 
Rapids had tenninees with the longest spells of unemployment and the most chance of having 
a physical handicap. 

D. Program Scale. Duration of Service and Cost 

Site Title II-A programs varied greatly in scale and average duration of enrollment, as 
shown in Table 2.9. The study site average for number of tenninees is below the national 
average because the study sites do not include any very large SDAs, 17 but on the other 
measures the averages for the study sites and the nation are similar. Differences in duration 
of service among the study sites are seen in the wide range of average number of weeks 
enrolled: for adults, this ranged from 7.67 in Jackson to 34 in Corpus Christi; for youths, the 
range was from 533 in Omaha to 33 in Corpus Christi. Program costs per adult terminee 
reflect both the length of enrollment and the higher service costs (based on higher rent and 
salaries) in large metropolitan areas such as Jersey City and Oakland. 

E. Site Performance Standards 

JTPA performance standards, as discussed in Chapter 1, assess program success in 
achieving a variety of program goals. Thus, it is important to ascertain whether the study sites 
are top performers under this system or include SDAs with the diversity of performance seen 
nationally. Tables 2.10 and 2.11 show that the study sites do include such diversity and as a 
group performed about the same as the national average. 

These tables show data for program year 1988, the year with the largest proportion of 
the 18-month study sample. For each of the three post-program outcomes listed, the tables 
show (1) the actual performance of the SDA, (2) the performance that was "predicted" by the 



17 As discussed earlier in this chapter, no SDAs with very few tenninees were included among the 
study sites, but this exclusion affected the average number of tenninees in the study sites less than the 
lack of veiy large SDAs. 
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TABLE 2.9 



SELECTED CHARACTERISTICS OF THE JTPA TITLE H-A PROGRAMS 
AT THE NATIONAL JTPA STUDY SITES 



Site 


Average Number 
of Terminees 
(PY 1987-89) 


Average Number of Months Enrolled 
Adults Youths 
(PY1987-89) (PY 1987-89) 


Federal Program 
Cost Per Adult 
Terminee ($) 
(PY1987-89) 


Butte, Moot 


576 


4.85 


4.46 


2,665 


Cedar Rapids, Iowa 


658 


7.24 


5.39 


2,212 


Cooaa Valley, Ga. 


1,063 


2.70 


3.54 


2,481 


Corpus Chris ti, Tex. 


1,049 


7.85 


7.62 


2,570 


Decatur, HI. 


525 


6.77 


5.77 


3,039 


Fort Wayne, Ind. 


1,195 


3.70 


7.16 


1,561 


Heartland, Fla. 


1,793 


3.54 


5.54 


1,782 


Jackson, Miss. 


1,227 




3.4o 


1,897 


jersey i^iry, im.j. 


Oft 

853 


3.62 


3.23 


3,637 


Larimer County, Colo. 


354 


7.39 


6.00 


1,937 


Marion, Ohio 


714 


6.24 


6.08 


2,199 


Northwest Minnesota 


430 


6.62 


6.54 


/ 1 


Oakland, Calif. 


1,396 


3.77 


3.93 


2,539 


Omaha, Neb. 


1,1U 


2.46 


2.85 


2,404 


Providence, R.I. 


503 


1.62 


1.23 


2,841 


Springfield, Mo. 


938 


4.00 


3.93 


1,898 


Site Average 


899 


4.63 


4.80 


2,377 


National Average 


1,177 


4.57 


4.97 


2,241 



SOURCES: JTPA Annual Status Reports (JASR) for program years 1987-89. 



NOTE: Data are averages for all JTPA Title H-A terminees during program years 1987-89. 
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TABLE 2.10 



JTPA PROGRAM OUTCOMES FOR ADULTS, BY SITE 





Adult Entered Employment Rate 




Actual 


rreuicteo ^aj 




Site 


(%) 




isinerence 


Butte, Mont. 


/**.u 


O/. 1 


0 


i^taMT icapias, low* 


76.9 


/ J.l 


J.o 


Coosa valley, ua. 




Oo.Z 


1 j.J 


Corpus cnnsti, icx. 


7Z.U 


O/. 1 




Decatur, 111. 


HCl A 

79.4 


OD.l 


14 1 


rort Wayne, ina. 


OA f\ 


77 A 


li.O 


Heartland, rla. 


HA C 

74.5 


Oo. / 




Jackson, Miss. 


67.6 


69.2 


-1.6 


Jersey City, N.J. 


86.S 


64.2 


ZZ.3 


Larimer County, Colo. 


68.0 


69.5 


-1.5 


Marion, Ohio 


55.5 


59.4 


-3.9 


Northwest Minnesota 


/J.J 


6Q 1 


4 4 


uiuuki, v*ain. 


67.4 


66.1 


1.3 


Omaha, Neb. 


65.0 


65.7 


-0.7 


Providence, R.l. 


74.3 


70.2 


4.1 


Springfield, Mo. 


89.0 


76.4 


12.6 


Site Average 


74.5 


68.2 


6.2 


National Average 


74.2 


67.3 


6.9 



SOURCE: JTPA Annual Status Report (JASR) for program year 1988. 

NOTES: There may be slight discrepancies in the reported differences of the averages 
because of rounding, and there may be small adjustments to the "rates that were 
made by states and not reported in JASR. 

(a) The predicted entered employment rate : based on the JTPA performar^c 
standard reported in JASR (PY1988). 
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DOL regression model used by most states to adjust the national performance standards to 
reflect the characteristics of local terminees and labor market conditions, and (3) the difference 
between actual and predicted performance. Youth positive terminations include job placement 
at termination, achievement of "employment competencies,* school completion, enlistment in 
the armed forces, and enrollment in other non-Title II-A training programs. On all three 
measures, the study sites include those that perform much better than their adjusted standard 
(those with large positive numbers in the difference column), some close to this standard, and 
some that fail to meet it (those with negative numbers). 

II. The Sample for This Report 

This section provides some basic background on the characteristics of the individuals in 
the research sample for this report The report's main focus is on the 11,474 members of the 
18-month treatment group sample (defined earlier in this chapter), since the primary goal of 
the analysis is to describe and analyze the nature of JTPA services provided in the study 
sites. 18 

Table 2.12 shows that 28 percent of the treatment group were under age 22 at random 
assignment, 32 percent were 22-29, 31 percent were 30-44, and 9 percent were 45 or older. 
Fifty-five percent are white, 30 percent black, and 12 percent Hispanic. Thirty-six percent 
were high school dropouts and had not passed the GED (General Educational Development, 
or high school equivalency) examination. Fifty percent had worked fewer than 13 weeks in the 
12 months before random assignment Twenty-eight percent were receiving some type of cash 
public assistance at application, and 12 percent had received Aid to Families with Dependent 
Children (AFDC) for two years or more. 

Behind these figures for the entire treatment group, there are differences among the 
target groups analyzed in this report As expected, males — both youths and adults — had a 
lower rate of cash assistance receipt at random assignment and the most recent work 
experience. Among adult women, the rate of receipt of cash assistance at random assignment 
(41 percent) was especially high, and 53 percent of this target group were single parents. Not 
surprisingly, adults were more educated than youths; 68 percent of adult men and 72 percent 



18 This analysis excludes the 5,557 control group members in the 18-mor.: 1 * study sample. 
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TABLE 2.12 



PERCENTAGE DISTRIBUTIONS OF SELECTED BASELINE 
CHARACTERISTICS OF TREATMENT GROUP MEMBERS, 
BY TARGET GROUP 





All Tiroet 




A Hul t 


IVMUC 


rxuuuc 


Characteristic 


Groups 


Men 


Women 


Youths 


Youths 


and Subgroup 


<%) 


<*) 


(%) 


<*) 


<%) 


Ace 












16-19 


17 2 


0 0 


0 0 


69 1 


J7.H 


20-21 


11 9 


0 0 


0 0 


^ /.y 


40 6 


22-29 


31.8 


44.6 


44.2 


0.0 


0.0 


30-44 


30.9 


43.3 


43.0 


0.0 


0.0 


45 ftnH nv^r 

MM UVvl 


8 0 
o.y 


il i 




n o 
u.u 


u.u 


Ethnicity 












White, non-Hispanic 


54.5 


57.0 


54.5 


53.7 


50.1 


ouiv&f uuu niMMiiic 


D\J.D 


9ft ft 


30 ft 


90 S 


DL.J 


Hfcnanir 

1U9UWUW 


11 0 

n.y 


y. / 


1 1 A 
11.** 




1 c o 
ID.o 


Other 


3.3 


4.5 


3.3 


2.1 


1.7 


Sex 












M. vlllilv 


ft 




ion o 

1UU.U 


n o 

U.U 


too 0 

1UU.U 


Male 


AS 9 


100 0 


0 0 


100 0 

1UU.U 


0 0 


Education 












High school diploma or GED 


63.5 


68.8 


71.8 


40.9 


50.8 


No hiffh school dinloma or GIiR 




31 2 


28 2 


59 1 


40 2 


Work Exnerience 












Worked 13 weeks or more 












in 12 months before assignment 


50.1 


58.5 


46.2 


53.0 


39.5 


Worked fewer than 13 weeks 












in 12 months before assignment 


49.9 


41.5 


53.8 


47.0 


60.5 


Public Assistance Receipt 












Not receiving AFDC or other cash 












assistance at assignment 


71.7 


83.5 


59.2 


87.6 


67.3 


Receiving AFDC or other cash 












assistance at assignment 


28.3 


16.5 


40.8 


12.4 


32.7 


AFDC History 












Never received AFDC 


73.2 


92.2 


49.6 


98.4 


71.5 


Received AFDC less than 2 years (a) 


14.5 


6.3 


22.8 


1.4 


21.3 


Received AFDC 2 years or more (a) 


12.4 


1.6 


27.6 


0.2 


7.2 


Household Composition 












Spouse present 


23.6 


35.8 


22.6 


10.5 


11.4 


No spouse present, child present 


28.7 


7.9 


53.2 


4.2 


35.5 


No spouse present, no child present 


44.7 


54.2 


20.7 


84.8 


51.0 



Sample Size 11,474 3,759 4,465 1,436 1,814 

(continued) 
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TABLE 2.12 (continued) 



SOURCE: MDRC calculations from Background Information Form responses. 

NOTES: Calculations for this table are based on data for all treatment group members 
in the 18-month study sample. 

Distributions may not total 100.0 percent because of rounding. 

(a) The AFDC receipt may not have been continuous. 
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of adult women were high school graduates or had passed the GED test compared with 41 
percent for male youths and SI percent for female youths. Later chapters will return to a 
more detailed discussion of the characteristics of this sample and how these characteristics 
affect service recommendations within JTPA and enrollment in the program. This analysis 
seeks to contribute to the ongoing national debate on how well JTPA server individuals with 
various key barriers to employment, men and women, and white and minority individuals. 

Reports by the U.S. General Accounting Office have analyzed JTPA services and 
outcomes for subgroups defined by gender, ethnicity, and measures of job-readiness including 
education, public assistance receipt, and recent work experience. 19 In light of this continuing 
interest in how JTPA serves the needs of minority and less employable applicants, subsequent 
chapters in this report (and the 18-month impact report) will analyze separate subgroups 
defined by ethnicity and employment barriers. 

Following a framework similar to these existing studies, this report identifies individuals 
as facing key employment barriers based on three factors: 

• education: individuals who have not received a high school diploma or passed 
the GED test at random assignment, 

• work experience: individuals who worked fewer than 13 weeks in the 12 
months before random assignment, and 

• public resistance receipt: individuals who were receiving AFDC or other 
cash assistance at random assignment 

Table 2.13 shows the proportion of the entire treatment group and of the target groups 
facing none, one, two, or three of these barriers to employment As was mentioned earlier in 
this chapter, many individuals in the sample do not face these barriers. In fact, 32 percent of 
the sample have none of these barriers, and another 38 percent have only one. 

Behind these summary numbers, however, there are important differences among the 
target groups. Adult men as a group face the fewest barriers to employment; 81 percent have 
one barrier or none. At the other extreme, only 58 percent of female youths have one barrier 
or none. 

Since subgroups based on the number of barriers to employment are analyzed in the 



"See US. General Accounting Office, 1989, 1990, and 1991. 
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TABLE 2.13 



PERCENTAGE DISTRIBUTIONS OF EMPLOYMENT BARRIER SUBGROUPS 
FOR TREATMENT GROUP MEMBERS, 
BY TARGET GROUP 



Number of 


All Target 


Adult 


Adult 


Male 


Female 


Employment 


Groups 


Men 


Women 


Youths 


Youths 


DUTierS \m) 




\70) 


\70) 


K 70 ) 


\ 70 ) 


None 


31.6 


40.3 


29.6 


25.9 


22.9 


1 


38.4 


40.5 


36.6 


42.4 


35.0 


2 


23.5 


16.3 


26.1 


26.8 


29.6 


3 


6.5 


2.9 


7.6 


5.0 


12.4 


Total 


100.0 


100.0 


100.0 


100.0 


100.0 


Simple Size 


11,474 


3,759 


4,465 


1,436 


1,814 



SOURCE: MDRC calculations from Background Information Form responses. 



NOTES: Calculations for this table are based on data for all treatment group members in the 
18-month study sample. 

Distributions may not total 100.0 percent because of rounding. 

(a) The barriers to employment include: not having a high school diploma or GED at 
the time of random assignment; having worked fewer than 13 weeks during the 12 months 
prior to random assignment; and receiving AFDC or other cash assistance at the time of 
random assignment. 
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remaining chapters of this report, Table 2.14 presents selected characteristics for all target 
groups and their employment-barrier subgroups. For example, the first panel of the table on 
age distribution indicates that 18 percent of those 16-19 years of age had none of the listed 
employment barriers, 36 percent had one, and 46 percent had two or three. In general, those 
with two or three employment barriers were more likely than the rest of the sample to be 
teenagers, females, and single parents. 

The next four chapters discuss the overall picture for the 16 study sites as a group in 
recruiting, enrolling, and serving JTPA applicants. This aggregated analysis describes the JTPA 
services received by the 18-month treatment group sample, for which impact estimates are 
calculated. Chapter 7 of this report returns to site-specific analysis, discussing the variation 
among the study sites on the topics discussed in the preceding chapters and seeking to link this 
variation to the features of individual sites discussed in this chapter. 
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CHAPTER 3 

BALANCING RESEARCH GOALS AND THE REALITY OF TTPA PROGRAMS 



The key issues discussed in this and the following chapter are interrelated To organize the 
discussion, this chapter first describes the goals of the National JTPA Study, which established 
the basic structure of the research design. The chapter then presents the steps that potential 
JTPA enrollees typically followed during program intake (labeled "client flow") in the study sites 
and how the research procedures were superimposed on the usual routines. With this summary 
of what actually happened in the sites during the study as background, Chapter 4 turns to a 
discussion of the competing policy questions that could be addressed, the resulting research 
designs, and the reasons for the approaches taken. 

This organization reflects the diverse audience for this report: employment and training 
practitioners, researchers, and policymakers. For researchers who may not be familiar with the 
JTPA system, this chapter provides background on "how the program works." For the employment 
and training community and policymakers, it provides a concrete example of "how random 
assignment works" and the questions the study addresses. 

I. The Goals of the National JTPA Study 

Before launching into a discussion of client flow and the research design, it is important to 
understand DOL's goals for the study. The central goal can be stated simply: to provide estimates 
of the difference that Title II-A of JTPA makes (i.e., its impacts) on the employment, earnings, 
and welfare receipt of those served in a diverse group of local programs. As part of this, DOL 
wished to estimate program impacts for key groups of clients, including the target groups (adult 
men, adult women, and male and female out-of-school youths) and, to the extent feasible, other 
subgroups based on ethnicity, education, work experience, and receipt of public assistance. 

To assess the impacts of this group of local JTPA programs, it is not possible merely to 
observe the post-program employment rates, earnings, and rates of welfare receipt (the program 
outcomes) for people who participate in JTPA because some of those served would have found 
a new job on their own or improved their skills and raised their income through other means 
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even if they had not participated in the program. 1 In a study of a voluntary program, the measure 
of a program's impacts is the difference between what happened to people who were given the 
option to participate in the program and what would have happened to them if they did not have 
this option. 2 This calls for creating and studying two groups of people with no systematic 
differences except their access to the program under study. The approach taken in this study, for 
the reasons given in Chapter 1, -was to randomly assign individuals who applied to JTPA to either 
a treatment group, which was given access to the program, or a control group, which was not 
given such access. As this and the following chapter will make clear, choosing the point in the 
JTPA application process at which random assignment is to occur involves balancing competing 
research goals, and the operational reality of JTPA programs. 

Additional goals of the study complicated thz task of designing and implementing the 
research. BOL sought to estimate both (1) the overall impact of JTPA Tide JJ-A services 
provided in each of a sample of local programs and (2) the specific impact of important categories 
of services such as those anchored in on-the-job training (OJT) and classroom training in 
occupational skills (CT-OS). Finally, DOL sought to study local programs that were following 
their normal practices as closely as possible. 

D. Client Flow and Study Procedures 

Figure 3.1 presents the basic steps common to all SDAs in the study by which an individual 
eUgible for JTPA could apply for the program, be randomly assigned, and (if assigned to the 
treatment group) enroll in the program. 3 Normally, many more people contact the SDA or a 
service provider to learn about the program and inquire about eligibility rules than actually apply. 4 
Consequently, as Figure 3.1 shows, individuals may "exit" from the intake process at each step. 
This could occur because they find other opportunities, decide JTPA is unlikely to provide what 
they want, or are discouraged by what staff tell them about their prospects in the program. This 



lThcsc tv P cs of outcome measures are used in the existing performance standards system, which plavs 
8 ""Xi! rolC fa JTPA mana 8 emcnt - Performance stanC-ds are discussed in Chapters l and 2. 

^Thc distinction between outcomes and impacts is not the satne as the distinction between short-term 
and long-term measures. Even long-term measures of program outcomes measure only the post-program 
status of the group served, not the difference the service made in their employment, earnings, and welfare 
receipt 

Appendix A presents profiles of the 16 study sites, including information about their recruitment 
process, assessment practices, and service providers. 

4 Some SDAs have tracked client contacts and report that about one quarter of those inquiring about 
the program ever enroll, with much of the drop-off coming early, before eligibility has been determined. 
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FIGURE 3.1 

FLOW OF SAMPLE MEMBERS IN THE NATIONAL JTPA STUDY SITES 



1ST AGE 1 1 



JTPA-Eligible Population 



kit"! 



Recruitment 






bat 


*■ 



Application 



Eligibility Determination 

* — 



Informed Consent Form 
and BIF Comp letion 
1 



Exit 



Exit 



Exit 



jSTAGE 2 



Classroom Training 
Service Strategy 



Random Assignment 



Assessment 



Recommendation of Services 
and Service Strategy 



OJT/JSA 
Service Strategy 



Random Assignment 



Exit 



Other Services 
Sen/ice Strategy 



Random Assignment 



Treatment 

Group 



Control 
Group 



jSTAGE j 



Treatment 

Group 



Control 
Group 



Enroll 



Do Not 
Enroll 



Receive 




Receive 




Receive 


Services 




Services 




No Services 



Treatment 
Group 



Control 
Group 



NOTE: Because of limited space, the stage 3 sample flow has been fully depicted only for the OJT/JSA service strategy, 
although it would be the same for the other two strategies. 
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section discusses these steps in the client flew shown in Figure 3.1 by grouping them into three 
stages. 

A. Stage 1. Connectine with Those gggflfe for and Interested in JTPA: 
Recruitment Application. Eligibility Determination, and RTF 
Completion 

!• The JTPA-eligible population. The federal statute defines the basic eligibility 
requirements for JTPA, which are principally low family income during the previous six months 
or current receipt of public assistance. 5 In each local area served through a single SDA, the 
size of the eligible population will vary with the characteristics of the population and local labor 
market conditions. However, JTPA is not a legal entitlement for all who satisfy the eligibility 
requirements. Nationally, JTPA funding is sufficient to serve fewer than 10 percent of those 
eligible, so the pool of eligible potential applicants normally greatly exceeds program capacity. 

2. Recruitment Nevertheless, recruitment of program applicants takes substantial effort 
in many (though not all) SDAs and can involve both the local adrninistrative entity running the 
program and agencies under contract to provide various kinds of training and employment 
services. Recruitment is a special challenge because SDAs must identify eligible individuals who 
are interested in the types of training and employment services that JTPA provides and are able 
to participate without receiving stipends or other program-related income support 6 For those 
without other sources of support, the program must offer income-generating services such as OJT 
or job search. 

The experience of program operators indicates that the level of unemployment ji the area 
is the best predictor of the difficulty of recruitment: the lower the level of unemployment, the 



JTPA authorizes the provision of job training for economically disadvantaged individuals and others 
facing serious barriers to employment An "economically disadvantaged individual" is defined as someone 
who (1) receives cash welfare payments under a federal, state, or local welfare program or is a member 
of a family that receives these payments; (2) has a total family income (excluding unemployment 
compensation, child support payments, and welfare payments) for the six-month period prior to application 
that, in relation to family size, is not in excess of the higher of (a) the poverty standard established by the 
Office of Management and Budget (OMB) or (b) 70 percent of the "lower living" standard, which is the 
standard above the poverty standard; (3) receives food sumps; (4) is a foster child on behalf of whom state 
or local government payments are made; or (5) as permitted by regulations, is an adult individual with a 
handicap, whose own income meets the famhy income requirements, exclusive of the income of other 
family members. 

One of the important changes made in the shift from CETA (the Comprehensive Employment and 
Training Act) to JTPA was that income support (in the form of regular stipends or public service 
employment) was virtually eliminated. Many SDAs do provide small payments to cover some expenses of 
participating (eg., transportation expenses), but in general participants must be able to support themselves 
throufh nonprogram resources. 
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harder it is to recruit applicants for JTPA. Because JTPA does not provide in-program income, 
as jobs become more plentiful JTPA becomes relatively less attractive. As evidence of this, even 
though low-income workers are eligible for JTPA, most applicants are unemployed or had 
previously left the labor force altogether. 7 

During the period of random assignment, sites had to identify a large enough pool of eligible 
applicants to be able to both seive the number of people they wished to enroll and to create a 
control group. With a random-assignment ratio of two treatment group members for each control 
group member, sites had to identify an "extra" 50 percent of eligibles. In most sites, this involved 
increasing recruitment efforts and expanding the applicant pool However, as mentioned earlier, 
the eligible population for JTPA remained much larger than the number of people recruited in 
the study SDAs. 

Because labor market conditions changed during the period of random assignment (in most 
sites unemployment rates declined) it is difficult to isolate the effect of this "study-induced" 
expansion in the applicant pool on the characteristics of applicants. However, program operators 
in the study sites reported that the combination of factors led them to recruit less job-ready 
applicants than had been the case in the raid 1980s, when unemployment was higher. 

3, Application and eligibility determination. Those who applied for JTPA completed the 
SDA or state forms used to establish their eligibility for the program. They were also required 
to provide documentation (usually information about recent family income or receipt of public 
assistance), the exient of which varied from state to state. SDA staff then had to determine if 
applicants were eligible for the program. Those found eligible would continue through to 
assessment, while ineligibles were told they could not be seived. 8 The study made no change in 
the eligibility determination process and staff were told to complete this process as they normally 
would. 9 

4. Completion of the Background Information Form (BIF), By this point in the process, 
applicants in the study sites typically had completed the study Background Information Form 
(BIF) with assistance from the program staff. The study design allowed some local flexibility as 



7 For example, 86 percent of the sample for the National JTPA Study were not working when they 
applied for JTPA See Bloom, 1991. 

8 Under JTPA rules, up to 10 percent of enrollees can have incomes above the normal eligibility cutoff 
if they have other barriers to employment 

9 As in most studies of ongoing programs, the research design did not include a supplementary check 
of eligibility. This is appropriate since the goal was to measure impacts of the program as operated 
normally for those on whom the local programs spent funds. 
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to the precise point when the form was completed to lessen the burden on local staff who 
assisted applicants in completing the form* In most sites, staff and applicants filled out the BIF 
when the usual SDA application materials were completed In some instances, completion of the 
BIF occurred as part of the assessment interviews. The BIF provided basic data on the pre- 
program characteristics of those randomly assigned, information that was used to identify members 
of subgroups analyzed separately in the study. 10 

B. Stage 2. Identifying the Research Sample: Assessment Recommendation 
for Services. Designation of a Service Strategy, and Random 
Assignment 

L Assessment Following eligibility determination (or possibly simultaneously for those 
very likely to be eligible), SDA and/or service provider staff assessed the current interests, skills, 
and service needs of applicants. The extent and complexity of this assessment varied greatly 
among the study SDAs; in some it consisted of a multi-day period of testing and interviews, while 
in others service recommendations were based on a short interview at the time the application 
was completed. Rural areas with limited service offerings, in which applicants may have had to 
travel long distances to the program office, were most likely to have combined the application 
with an abbreviated assessment. 

During the study, local staff were told to follow past practices as closely as possible, and in 
most SDAs assessment practices were not changed by the study. In some SDAs, however, 
modifications were made because of locally initiated changes in program design (e.g., changes to 
streamline intake practices) or because staff were seeing less job-ready clients, as noted above. 

2. Recommendation for services. Based on their assessment of clients, local staff 
developed recommendations for services reflecting client needs, interests, and preferences. For 
about three quarters of the 18-month treatment group sample, staff recommended a single service 
(most commonly CT-OS or OJT), but for the remaining one quarter, staff recommended a 
combination of services. 11 

For some applicants, staff decided that no JTPA services would be appropriate and - as 
would normally be the case - these individuals did not continue through the remaining steps of 



10 These data were used in the analysis of the characteristics of the 18-month treatment group sample 
in Chapter 2, later analysis of subgroups in this report, and the analysis of impacts for key subgroups in 
the 18-month impact report. They were also used for the baseline characteristics report published earlier 
in the project (Bloom, 1991). 

n The details of service recommendations are discussed later in this chapter. 
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the JTPA intake process. This judgment could be made for several reasons: the applicants' skills 
were too poor to meet entrance requirements for JTPA services; staff believed applicants had 
other problems (such as substance abuse or mental health issues or "poor motivation") that posed 
serious obstacles to success in the program; applicants were so employable that the SDA did not 
want to spend program funds serving them; or the SDA did not offer the type of service that the 
applicants sought 12 Applicants whom local staff believed to be inappropriate for JTPA did not 
become part of the research sample; the study did not, therefore, require staff to serve them. 

3. Designation of a service strategy. Since this aspect of the research design is complex, 
some background on its origins is useful In the planning stage of the study, DOL set as a goal 
the estimation of program impacts for important categories of services. In addition, in 
implementing the study, DOL also wished to change local program operations as little as possible. 
Ilierefore, the research design had to define the study's service categories or strategies carefully 
to reflect actual program practice. 13 A central objective was to develop separate categories for 
individuals recommended for the two most commonly provided, more intensive services under 
JTPA: CT-OS and OJT. 14 But the situation was complicated by the fact that, in a substantial 
minority of cases, lo^al staff typically recommended more than one service. 

The resulting research-defined service strategies, therefore, distinguished between these two 
emphases or primary services, but also accommodated combinations of services. Individuals 
recommended by local staff for CT-OS but not OJT were part of the service strategy labeled 
classroom training (CT). Those recommended for OJT but not CT-OS were part of the service 



12 The issue of screening for motivation has been a contentious one in JTPA. Some SDAs have 
deliberately set up application procedures that require people to return several times to complete forms 
or attend interviews in an effort to screen out those without the motivation to persevere. Others argue 
that such approaches impose many costs on applicants (in terms of time) before the benefits of program 
participation are apparent and result in programs losing many applicants who could have been helped. 
See Kelly, 1987, for one view of this issue. 

13 TTiis mesh of research-defined concepts and actual program practice was especially important 
because, as discussed later in this chapter, staff had to designate a service strategy for each individual prior 
to random assignment and then follow a service plan consistent with this designation for individuals 
randomly assigned to the treatment (or program) group. In effect, staff were not permitted to change their 
mind after random assignment The reason is discussed briefly later in this chapter and in Doolittle and 
Traeger, 1990. Consequently, it was important to develop service strategy definition! that would inhibit 
local discretion as to services as little as possible and still produce patterns of post-nmdom-assignment 
service receipt that represented real differences in emphasis. 

14 At the time the study started, data from the National Job Training Longitudinal Survey (JTLS) 
showed that about 75 percent of JTPA enrollees nationally were enrolled in only one service and slightly 
under 50 percent were enrolled in either CT-OS or OJT. During the period of study sample build-up, the 
proportion of enrollees in these two services increased to about 60 percent. 
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strategy labeled on-the-job training/job search assistance (OJT/JSA). This name was chosen 
because over the course of random assignment about one third of individuals in this category 
were recommended for both of these services designed to get individuals immediately into a job. 
Those recommended for neither CT-OS nor OJT, or for both services, were part of the third 
strategy labeled other services. 

Table 3.1 shows the services allowed under each of the three service strategics used in this 
implementation analysis and the 18-month impact report. Only to#u restrictions on SDA and 
service provider actions are implied. First, individuals designated for the classroom training 
service strategy (Le., recommended for CT-OS as their primary service) were not later to receive 
OJT, and those recommended for the OJT/ISA service strategy were not to receive CT-OS. 15 
This restriction seemed to be consistent with operational reality; SDA experience suggested that 
clients tend to divide into (1) those interested and abte (in terms of educational attainment, 
learning style and study habits, and a source of income while in training) to participate in 
ciassroom-based occupational training and (2) those seeking immediate employment because of 
financial obligations or an aversion to the classroom setting, but also in need of training. Thus, 
these service strategies accommodated the needs and interests of most JTPA applicants, and 
client preferences were an important factor in assessments in most SDAs. In addition, SDAs 
rarely provided both CT-OS and OJT to a single client, since the combined cost of these two 
services exceeded what most JTPA programs were prepared to spend on one person. 

The second restriction on SDA actions was a ceiling on the proportion of clients who could 
be designated for the other services strategy. Since this strategy contained no restrictions on 
subsequent service receipt, SDAs would have had an incentive to designate everyone for this 
strategy. Consequently, the research team negotiated with each site a ceiling on the strategy, with 
the level reflecting past patterns of service receipt. For example, in SDAs where most individuals 
typically received only OJT or CT-OS, the ceiling on other services was set low; in SDAs where 
many individuals received a combination of CT-OS and OJT, or services other than these two, 



rtPP Pfn-«*y through the study, the research team agreed to allow SDAs to provide a small amount of 
OJT fallowing CT-OS in cases where it was needed to secure placement in a job. This OJT could not 
amount to more than 20 percent of total training time. Similarly, SDAs were allowed to provide a small 
amount of CT-OS (again, no more than 20 percent of the training time) when needed to give the applicant 
a skills brush-up" in order to secure an OJT position. As the service enrollment data presented later in 
this report show, this option was used very rarely. • 
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TABLE 3.1 



PROGRAM SERVICES AVAILABLE IN THE THREE 
SERVICE STRATEGIES 



Classroom Otter 
Program Service Training OJT/JSA Services 

Classroom Training in 



Occupational Skills (CT-OS) 


Yes 


No (a) 


Yes 


On-tne-Job Training (OJT) 


No (a) 


Yes 


Yes 


Job Search Assistance (JSA) 


Yes 


Yes 


Yes 


Basic Education (BE) 


Yes 


Yes 


Yes 


Miscellaneous Services (b) 


Yes 


Yes 


Yes 



NOTES: (a) In general, this program service was not available, but, as discussed in the 
text, there were exceptions. In certain circumstances, a short CT-OS "brush-up" course 
was allowed prior to OJT in the OJT/JSA service strategy, and brief OJT could follow 
CT-OS in the classroom training service strategy. 

(b) Miscellaneous program services include such activities as assessment, job- 
readiness preparation, vocational exploration, job shadowing, and tryout employment. In 
this table, miscellaneous program services also include work experience, which is 
identified as a separate program service in other reports on the National JTPA Study. 
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the ceiling was set much higher. Among the 16 SDAs in the study, the range was from 15 
percent to 70 pcrceni, with more than half the sites at 35 percent or lower, 16 

4. Random assignment to tfcc treatment or control group. The random-assignment 
procedure was straightforward Site staff called MDRC (using a toll-free number), provided 
background information on each individual to allow tracking of sample build-up by target group, 
and listed the service recommendations and the service strategy designation. MDRC staff gave 
site staff the person's research status (treatment or control); they were then to follow their 
service recommendations for those randomly assigned to the treatment group. MDRC staff kept 
a roster of those randomly assigned and their research status to assure that if a person in the 
sample reapplied to the program, he or she would be treated consistently during the follow-up 
period 

Those in the control group were excluded from JTPA-funded services in the SDA for the 
following 18 months, but were provided with a list of alternative service providers in the 
community whom they could contact on their own. 17 Detailed information presented in Appendix 
B shows that sites were quite successful in enforcing this rule. About 3 percent of the controls 
were enrolled in any JTPA services during the follow-up period. 18 

Essentially, as shown in Figure 3.1, this process created a separate control group for each 
service strategy, permitting separate impact estimates for how each strategy worked for the people 
recommended for it The process also permitted a combined impact estimate for the entire 
sample. As will be descubed later in this report, local staff recommended people with different 
characteristics for the three service strategies, so at random assignment the sample in each 
strategy varied in important ways from the others. Because of this, it is not possible to compare 
the effectiveness of the three service strategies for similar groups of people. The treatment and 



Over the course of random assignment, most sites recommended many fewer applicants for the other 
services strategy than their ceiling allowed. Overall, the proportion of the 18-month treatment group 
sample in this strategy was 27 percent 

17 After 18 months, members of the control group could receive JTPA services if they returned to the 
SDA or service provider on their own. However, SDA staff agreed not to make any special effort to 
recruit members of the control group at the end of their exclusion from JTPA 

18 Some members of the research sample were participating in a title of JTPA not included in the 
study (e.g., the summer youth program) prior to random assignment and had applied for Title II-A services. 
Those assigned to the treatment group were given access to Title II-A services, but those assigned to the 
control group were not. Hdwever, since control group members were allowed to finish their non-Title II- 
A services, they would appear as enrolled in JTPA at the beginning of the follow-up period. Of the 159 
members of the control group with JTPA enrollment during the follow-up period, 60 people began their 
enrollment before random assignment, mostly in non-Title II-A services. Thus, less than 2 percent of 
controls began a JTPA enrollment after random assignment 
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control groups in each strategy differed in initial characteristics arJ in the type of service 
recommended by staff, making it impossible to isolate the effect of the service strategy on 
impacts. 19 

C Stage 3. Efforts to Arrange Services for the Treatment Group: Enrollment 
in JTPA and Involvement of the NonenroHed with JTPA 

Individuals randomly assigned to the treatment group were offered access to JTPA services; 
this offer and the resulting services provided were the "treatment" being tested through the 
random-assignment experiment In effect, this design was testing the impact of the decisions that 
local staff and individual applicants made during the intake process: whether to tiy to arrange and 
participate in JTPA services. As will become clear in the following discussion, many factors 
affected whether an individual actually ended up enrolled in JTPA Also, arranging services 
involved considerable administrative effort and expended program resources; thus, starting the 
impact story at the point of referral to services makes sense. 

As will be explained later, the impact analysis in the National JTPA Study will also present 
alternative impact estimates that seek to capture the difference that participation makes. 
However, as will also be clarified later, the concept of "participation" in JTPA is somewhat vague 
and hard to measure, complicating the interpretation of they; alternative impact estimates. 

1. Enrollment in JTPA services. Enrollment in JTPA occurs when SDA staff enter a 
person's name and application data into the local JTPA management information system (MIS) 
and enroll her or him in one or more specific JTPA-funded services. This step makes the person 
an official JTPA participant, whose service receipt and progress are tracked and whose 
termination status (e.g., employment and wages on leaving the program) is noted as part of the 
JTPA performance standard system. By enrolling clients, SDA staff are held accountable through 
the JTPA performance standard system for the costs that JTPA incurs in serving them and for 
their success when they leave the program. 20 As discussed in more detail in Chapter 4, 64 



19 This issue is discussed in more detail in Chapter 4. 

20 As discussed in Chapter 1, during the period of the study, clients' employment was measured as of 
their point of termination from the program. Suiting in program year 1990, performance standards were 
amended to include measures of employment 13 weeks after leaving the program. Controversy over the 
costs standards led DOL to end their use in program year 1990. One major concern was whether a focus 
on costs per placement for adults or per positive termination for youths provided too many incentives to 
run short-term, low-intensity programs. A second concern was that the focus on JTPA program costs could 
provide a distorted cross-site comparison, since local programs vaued in their ability to draw on other 
funders to provide services for JTPA clients. For example, in some SDAs it was relatively easy to get 
clients into basic education funded by state education programs, while in others this was difficult. 
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percent of the 18-month treatment group sample were enrolled in JTPA at some point in the 18- 
month follow-up period Thus, Egurc 3.1 shows individuals who enrolled in JTPA as a subset 
of each of the three treatment groups. 

2* Noncnrollcd treatment group members . Four factors help explain why some members 
of the treatment group would never be enrolled in JTPA. 

a. Despite the initial assessment that a client is appropriate for JTPA, 
staff may be unable to find a service provider willing to accept the 
person. Service providers often have the final say on whether they 
will accept an applicant Many classroom training agencies have 
entrance requirements on minimum initial skills, and employers who 
can provide OJT will typically make their own assessment about 
whether they want the applicant as an employee. In addition, an 
applicant may be seeking classroom or other training at a time of the 
year when it is not offered. 

b. Applicants may change their mind about JTPA as they continue to 
seek other opportunities or learn more about the program. Many are 
looking for work on their own, and some will find employment Some 
may discover different ways to finance the type of training they seek. 
And still others may decide that they are not interested in a job or 
training after alL 

c The design of the JTPA program encourages local staff to make sure 
that applicants are going to participate and do well in a service 
before they are enrolled find counted as a JTPA participant The 
high visibility of standards within the JTPA system go well beyond 
their limited role in allocating incentive grants, consisting of up to 6 
percent of Title II-A funds. An SDA's success in exceeding its 
performance standards is often seen as a sign of how well a program 
is operating. This clearly encourages SDAs to focus on achieving 
their standards on the various outcomes measured, but it also creates 
an incentive to hold off on enrolling individuals (i.e., having them 
count as part of the program) until they are placed in and begin a 
service that staff feel is likely to produce success. 21 

d. Many SDAs believe that they have discretion in defining the point 
at which individuals "count 91 in their performance standards and 

respond to the incentives of the standards by delaying enrollment In 
the initial years of JTPA DOL adopted the position that JTPA was 
to be primarily controlled by states and localities. Therefore, DOL 



-"Although the performance standard system has changed in recent years to include longer-term 
measures of success (i.e., 13 weeks after leavin ■» the program) and measures less closely tied to immediate 
employment and low costs, the basic incentives remain unchanged. SDAs can be seen as well run and gain 
some additional funding if a high proportion of people leaving their program find a job that pays well or 
have attained a variety of employability-enhancing competencies. 
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did not define precisely many key administrative terms, including the 
point at which enrollment should occur* In recent years, this federal 
stance has changed, but a holdover from the initial period is the 
continued practice of linking enrollment to the actual beginning of 
the intended service, be it the first day of class attended, the first 
workday for OJT, or (in an extreme example of this) when 
participation in job club or job search assistance leads to employment 

There is no national survey that has collected information on the point of enrollment in 
local JTPA programs. However, most of the study sites enrolled individuals in classroom training 
when they attended their first class or in OJT when they worked their first day, though one site 
did enroll people during assessment In a few sites, local staff could refer people to job search 
assistance or job club without enrolling them and observe how they acted in this setting as part 
of an "extended assessment" The applicants may never have been enrolled in JTPA unless they 
found a job or were referred to another service because their behavior in the job club showed 
motivation and promise of employability. 22 

These factors suggest that nonenrolled members (as well as enrolled members) of the 
treatment group could have had some post-random-assignment involvement with the JTPA system. 
As will become clearer in Chapter 4, this could complicate the calculation of impacts per person 
"participating" in JTPA because enrollment and receipt of some JTPA assistance may not be 
synonymous. In order to understand the extent to which this occurred, the research team drew 
a sample of nonenrolled treatment group members in 12 sit^s 23 and talked with local staff about 
their efforts to work with these individuals after random assignment. 

Table 32 presents the findings from this special study. The local staff had no contact with 
15 percent of this sample after random assignment; basically, they were unable to locate them 
again. Another 11 percent reported that they were no longer interested in JTPA, for a variety 
of reasons. Another 20 percent of the sample were recontacted, but staff never arranged service 
for them. The remaining S3 percent of the sample of nonenrolled treatment group members had 
some post-random-assignment involvement with JTPA without being enrolled. The most common 
service, provided for 36 percent of the sample, was one or more referrals to employers for a 



^Legislation proposed by DOl and by members of Congress in 1991 to amend JTPA woultf tighten 
some of the definitions of key administrative terms, in recognition of this problem. In addition, DOL has 
recently issued new guidelines defining more precisely key administrative terms. 

2 %ie samples were drawn in two time periods: November 1988-Januaiy 1989 and March-June 1989, 
ard included most of the SDAs where random assignment was occurring at those times. Within these time 
periods, random samples of nonenrolled tmtment group members were drawn. 
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TABLE 3.2 



PERCENTAGE DISTRIBUTION OF 
POST-RANDOM ASSIGNMENT ACTIVITY IN JTPA 
OF TREATMENT GROUP MEMBERS WHO DID NOT ENROLL 



Nonenrollees 



Activity (%) 

No Further Contact 15 

Further Contact, But Not Eligible l 

No Longer Interested (a) 11 

Got job on own 5 

Moved 2 

Health problems 1 

In another program 1 

Reason unknown 3 

Interested, But Made Contact Only 

and Received No Services 20 

Interested and Received Service(s) (b) 53 
Received further assessment 

and counseling 11 
Referred to classroom 

training providers) 5 

Received support servicers) 2 
Referred to employees) for 

possible on-the-job training 36 
Participated in job club or 

received job search assistance 20 

Total 100 



Sample Size 307 



SOURCE: Information collected by MDRC site representatives during the National 
JTPA Study. 

NOTES: Calculations for this table are based on data for a random sample of 
307 treatment group members in the 18-month study sample who did not enroll in 
JTPA. 

(a) When totaled, the sub ^itegory percentages are over 1 1 percent because 
nonenrollees could cite more than one reason for no longer being interested in JTPA. 

(b) When totaled, the subcategory percentages are over 53 percent because 
some nonenrollees received more than one service. 
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possible OJT position. Twenty percent of the sample participated in job club or other job searcii 
assistance. This small study suggests that local staff worked with about half of the treatment 
group members who never enrolled in JTPA, though in many cases little service was provided 

The results from this survey suggest that local staff worked (with a wide range of intensity 
and commitment) with about 80 percent of the treatment group. This 80 percent consisted of the 
64 percent who were enrolled, plus an additional 18 percent (i.e., half of the 36 percent of 
nonenrollees) who were never enrolled but did have some post-random-assignment JTPA 
involvement 24 



24 This finding does not relate directly to the ongoing debate about how much local JTPA programs 
"cream" ir selecting enrollees, i.c, seek more employable clients and shy away from enrolling harder-to- 
serve Clients. Much of this creaming, if it existed, could have occurred before the point of random 
assignment, the point at which the 18-month treatment group sample was identified. Random assignment 
followed local staffs assessment of clients and a decision that they would be appropriate for the program. 
Nevertheless, as analysis presented in Chapter 7 shows, there were differences in the rate at which SDAs 
in the study enrolled the subgroup in the sample with the most barriers to employment. There were also 
some differences in overall enrollment rates among subgroups in the sample defined by ethnicity, education, 
gender, and other personal characteristics. 
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CHAPTER 4 



KEY POLICY QU ESTIONS AND THE RESULTING STUDY DFSffiN 

This chapter - building on Chapter 3's description of the typical flow of potential JTPA 
enrollees - discusses how competing study objectives presented issues that had to be addressed 
in developing the research design. The discussion focuses on three key research design issues: 
choosing the random-assignment model, providing alternative program impact estimates, and 
defining the service strategies analyzed in the study. In addressing this third issue, the chapter 
presents information on the services received by treatment group members recommended for the 
three service strategies analyzed. 

I. Choosing the Random-Assignment Design 

The choice of the random-assignment design determines the questions that the research can 
address. DOL's multiple goals for the study presented a challenge in developing the research 
design. These goals, in brief, were to estimate overall impacts for JTPA programs in the sites, 
to estimate impacts for specific service strategies anchored in classroom training in occupational 
skills (CT-OS) and on-the-job training (OJT), and to assess the impact of local programs that were 
following their normal practices as much as possible. 

The first two goals (overall program net impacts and specific service strategy net impacts) 
could both have been accomplished using the research design presented in Figure 4.1, where 
random assignment occurs after eligibility determination, but before local staff have an opportunity 
to recommend services. Under this approach, applicants who are found by local staff to be 
eligible for JTPA and appropriate for its services through a preliminary assessment would be 
randomly assigned to one of three service strategies or to a control group. This design could 
produce three types of impact estimates (the two required net impacts plus another type of 
estimate): 

1. Overall program impacts: Comparisons of the post-random-assignment 
experiences of the three treatment groups with the control group would produce 
estimates of the impact of the overall program. 

2. Service strategy impacts: Comparisons of those in the treatment group for a 
service strategy with the control group would produce estimates of the impact 
of that service strategy. 
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FIGURE 4.1 

POTENTIAL DESIGN FOR RANDOM ASSIGNMENT 
TO A SERVICE STRATEGY OR TO THE CONTROL GROUP 
(NOT USbD IN THE NATIONAL JTPA STUDY) 
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3. Comparisons of the relative (or "differential*) impacts of service strategies: 
Because assignment io each of the three treatment groups and to the control 
group would have been random, these groups would initially have been similar 
at random assignment except for the differences in access to JTPA services. 
This would permit comparisons of the impact of one service strategy with 
another 

While this approach has advantages, it is clearly inconsistent with DOL's third objective: to 
study local programs that were following normal JTPA practices insofar as possible. Random 
assignment of applicants to service strategies would override the normal assessment and service 
recommendation prrctices of SDAs and prevent the estimation of impacts for those who normally 
receive the various services. Since individualized service plans based on client interests and needs 
are one objective of JTPA, this research design was not acceptable. 

Instead, the research team sought to introduce random assignment into the application 
process in a less disruptive way while st»!i producing (1) an estimate of the net impact of overall 
JTPA services in the study sites and (2) estimates of the net impact of each service strategy for 
the people local staff found appropriate for it In the resulting research design, presented in detail 
in Figure 3.1 and summarized in Figure 42, random assignment follows local staff 
recommendations for services and designation of a service strategy. 

This design addresses the two central impact questions: 

1. Overall program net impacts are estimated by combining the three treatment 
groups (recommended for the classroom training, OJT/JSA, and other services 
strategies) and comparing their experience following random assignment with a 
combined control group made up of the three separate control groups. 

2 Service strategy net impacts are estimated by comparing the experiences of the 
treatment gro, p for each strategy with those of the control group for that 
strategy. 

This strategy also preserves to a much greater extent local staffs ability to operate their programs 
following normal practices as to intake, assessment, and rervice referral However, the research 
design used in this study does not produce experimental comparisons of the relative (or 
"differential") impacts of service strategies because assignment to each of the three treatment 
groups and the control group was not random. Therefore, those referred to the three service 
strategies differed initially on many characteristics in addition to the services to which they were 
given access. This choice of research design rests on an implicit decision by DOL that testing 
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JTPA as closely as possible to the way it normally operates was more important than producing, 
through the experiment, differential impacts for the service strategies. 

Once this choice was made, the research team faced the issue of when in the client flow 
to do random assignment, which also involved balancing research goals. The goal of estimating 
net impacts leads to a desire to place random assignment as early in the intake process as possible 
in order to lessen the control group's contact with the program. Yet local staff had to work with 
a client long enough to make recommendations for services and designate a service strategy 
before random assignment 

Another factor also was relevant: as mentioned in Chapter 3's discussion of client flow, at 
each stage in program intake applicants can "exit" the system. Therefore, the closer random 
assignment is placed to the actual start of services 0\e., the later in the intake process), the 
higher the proportion of the treatment group receiving services. This makes the comparison of 
the experiences of the treatment group to the control group closer to a test of the impact of 
service receipt (which would be preferred) as opposed to the offer of services. 

The choice made for the research design placed neater emphasis on producing net impacts 
of JTPA (i.e., a control group vith little involvement with the program) than on producing high 
enrollment rates in specific JTPA services. The research team worked with staff in each site to 
identify an early point in JTPA client flow at which local staff believed they could decide whether 
an applicant was appropriate for JTPA and identify the appropriate services and service strategy. 
In most cases, this came at the end of the normal assessment, which typically occurred quite early 
in the application process. In one site, staff decided to perform an initial assessment of clients 
earlier than it normally did. 1 

H. Providing Alternative Program Impact Estimate 

These decisions on the point of random assignment increased the importance of providing 
alternative impact estimates: 

• a pure "experimental" comparison of the treatment group and the control 
group, which measures the impact of providing access to JTPA services to those 
people who local staff recruited, assessed, and found eligible and appropriate 
for the program; and 



^This occurred in Larimer County, Colorado. See Appendix A for the details of client now and 
random assignment in this site. 
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• a conversion of this estimate to an approximate measure of the impact of JTPA 
on people who actually received services; Le., adjusting for the fact that some 
treatment group members never received program services* 

Calculating the pure experimental impact estimate is relatively straightforward: this estimate 
is the difference in the average outcomes for treatment group members compared with the 
averages for control group members. The outcomes measured in this study include, among others, 
employment rates and earnings. These impacts are labeled "impacts per assignee 91 in the 18- 
month impact report and stand for "impacts per person randomly assigned to the 18-month 
treatment group sample." 2 This treatment-control group comparison rests solely on the validity 
of random assignment, which assures that there were no systematic differences between the two 
groups at random assignment except that the treatment group was then given access to JTPA 
services and the control group was not 

Ideally, one would convert this measure of the impact of the offer of access to JTPA to 
an estimate of the impact of receiving JTPA services, i.e., "impacts per service recipient" 
However, two types of issues arise in this effort 

On the data side, the only measure of service receipt in JTPA available for the entire 
sample comes from the local SDA enrollment records. The results reported earlier on the 
involvement of nonenrolled treatment members with JTPA imply that enrollment records 
understate the receipt of JTPA assistance. The small sample size of this special study of 
nonenrolled treatment group members makes it impossible to use these data to develop a more 
accurate measure of service receipt for the full sample; thus, enrollment has to be used as a 
proxy measure of "participation" in the program, even though it is likely to understate service 
receipt. 

Putting aside these data problems, there is no technique for directly calculating impacts per 
enrollee that is as reliable as the pure experimental calculation of impacts per assignee. Enrollees 
in JTPA are not a random sample of all treatment group members so it is incorrect to compare 
the experiences of enrollees with those of the entire control group. Since there is at this time 
no reliable way to identify the control group counterparts of people in the treatment group who 
ended up enrolling in JTPA, a different approach must be tried. 3 



2 Bloom et al , 1993. 

^uch attempt t encounter the types of analytical difficulties that earlier nonexpcrimental impact studies 
experienced, which led DOL to decide on a random-assignment impact study. The nonexperimentai 
component of this project is attempting to develop new methods to address these problems. 
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The precise details of the underlying assumptions and the calculations used to convert 
impacts per assignee to impacts per enrollee are discussed in the 18-month impact report, but the 
approach used assumes that all program impacts accrue to those who participated in JTPA (i.e., 
that nonparticipants were unaffected by JTPA) and that enrollment is a good measure of 
participation* Impacts per enrollee are, therefore, "scaled up" impacts per assignee, reflecting the 
assumption that enrollees are the source of all program impacts. As an illustration of this 
conversion from impacts per assignee to impacts per enrollee, if the program affects only 
enrollees, and if half of the treatment group enrolls in the program, then impacts per enrollee will 
be twice those per assignee. 4 

This implied impact per enrollee rests on an un testable assumption that nonenrollees are 
unaffected by the program. However, the finding of nonenrolled service receipt, reported in 
Chapter 3, illustrates the need to question this assumption. 

Recognizing this need for caution, the 18-month impact report presents impacts per assignee 
and per enrollee, but argues that these "bracket" the ideal estimate: impacts per service recipient 
To illustrate why this is true, assume again that 50 percent of the treatment group were enrolled, 
so that a conversion from impacts per assignee to impacts per enrollee would double the size of 
the impacts. However, if an additional 25 percent of the treatment group, for example, received 
some JTPA service without enrollment, then 75 percent of the treatment group were actually 
service recipients (50 percent enrolled, 25 percent not). Impacts per service recipient would be 
only 50 percent larger than impacts per assignee. 

The only assumption needed for this bracketing argument to hold is that any program 
impacts on nonenrolled "service recipients" do not have a different sign than the impacts on 
enrollees. For example, if program impacts on enrollees are positive, the impacts on nonenrollees 
must be zero or positive. If this assumption holds, the "intensity" of the services received by 
nonenrollees and the size of the resulting impacts determine the "width" of the bracket: typically, 
the less intensive the services, the smaller the bracket 

The findings on nonenrolled involvement in JTPA, reported in Chapter 3, are generally 
consistent with this view, since most of the service for the half of the sample receiving some 
JTPA assistance is clearly not intensive. However, in instances where individuals are referred to 



*The conversion is made by dividing impacts per assignee by the enrollment rate. In this example, 
division by JS doubles impacts. See Bloom et aL, 1993, for the details of this procedure. 
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job club, they could have participated in the same services that led to enrollment for those who 
found a job. 

HI* Defining Understandable and Operationally Relevant Service Strategies 

The third major research issue confronted in the study was the need to define service 
strategies for the analysis that allowed separate analysis of key service emphases (classroom 
training and OJT) while modifying normal assessment and service delivery practices as little as 
possible. The discussion in Chapter 3 on the client flow in JTPA described the definitions of 
service strategies that local staff used as part of the random-assignment process. A proof of the 
usefulness of the definitions comes not from their theoretical attractiveness, but rather by 
examining the patterns of service recommendations and enrollment for the individuals randomly 
assigned to see if these desired service emphases actually did emerge as the study was 
implemented. 

To summarize the conclusion of this section: treatment group members in the classroom 
training service strategy clearly received services that emphasized basic education and CT-OS, 
with relatively high enrollment rates and a high percentage of enrollees receiving some type of 
classroom-based services. Treatment group members in the OJT/JSA service strategy had lower 
rates of enrollment in JTPA, but again a high percentage of enrollees received one of these 
services. As intended, the other services strategy produced a diverse collection of services. There 
was great variation among the sites in service emphasis and enrollment rates, topics discussed in 
Chapter 7 of this report 

Table 4.1 shows the service recommendations made by local staff for the 18-month treatment 
group sample, with individuals grouped into the three service strategies defined for this study: 
classroom training, OJT/JSA, and other services. For all groups (shown in the first column), staff 
recommended the classroom training service strategy for 36 percent of the individuals, OJT/JSA 
for 37 percent, and other services for 27 percent. Under each service strategy subheading, the 
rows show the percentage of the sample that staff recommended for the individual services and 
combinations of services. For example, staff recommended 32 percent of all groups for CT-OS 
as a sole service, 1 percent for CT-OS and job search assistance, and so forth. 

The pattern of service recommendations varied greatly among the four target groups shown 
in the remaining columns of Table 4.1. Staff recommended the classroom training service strategy 
for 44 percent of adult women and female youths compared with only 25 percent of adult men 
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TABLE 4.1 



PERCENTAGE DISTRIBUTIONS OF SERVICE STRATEGY RECOMMENDATIONS 
AND RECOMMENDED PROGRAM SERVICES, 
BY TARGET GROUP 





All Target 


Adult 


Adult 


Mile 


Female 


Recommended S*rv«« Strategy 


Groups 


Men 


Women 


Youths 


Youths 


and Program Services 


(%) 


(«) 


(*) 


(%) 


(%) 


Classroom Training 


35.9 


Oil £ 

Z4.0 


AA A 

44. U 




AA O 

44.3 


CT-OS only 


31.5 


21.4 


38.6 


26.4 


39.3 


CT-OS and JSA 


0.8 


0.6 


1.3 


0.2 


0.7 


CT-OS and BE 


0.6 


0.2 


0.5 


1.2 


1.2 


CT-OS and misc. services (a) 


0.2 


0.1 


0.2 


0.3 


0.5 


CT-OS and 2 or more 












other services 


2.7 


2.4 


3.4 


1.7 


2.6 


OJT/JSA 




48 7 






23 1 


OJT only 


24.0 


29.8 


23.6 


19.9 


16.0 


OJT and JSA 


8.1 


12.2 


7.0 


6.5 


3.7 


OJTandBE 


0.2 


0.2 


0.0 


0.5 


0.1 


OJT and misc. services (a) 


0.6 


0.1 


1.1 


0.6 


0.7 


OJT and 2 or more 












other services 


4.5 


6.5 


3.2 


5.4 


2.7 


Other Services 


26.7 


26.7 


21.0 


37.3 


32.5 


JSA only 


5.8 


8.6 


5.2 


4.9 


2.0 


BE onlv 


2 7 


0 4 


1 i 

A » 1 


8 5 


6.9 


Misc. services only (a) 


13.2 


14.4 


10.0 


16.5 


15.7 


CT-OS and OJT 


0.3 


0.1 


0.4 


0.3 


0.3 


JSA and BE 


0.1 


0.1 


0.0 


0.1 


0.1 


JSA and misc. services (a) 


1.6 


1.9 


1.6 


1.4 


1.3 


BE and misc. services (a) 


1.1 


0.2 


0.3 


3.6 


3.3 


3 or more services 


2.0 


1.0 


2.4 


1.9 


2.9 


Total 


100.0 


100.0 


100.0 


100.0 


100.0 


Sample Size 


11,474 


3,759 


4,465 


1,436 


1,814 



SOURCE: MDRC calculations from Background Information Form responses. 

NOTES: Calculations for this table are based on data for all treatment group members 
in the 18-month study sample. 

Recommended service strategy distributions may not total 100.0 percent 
because of rounding. Recommended program service distributions may not total the percentage 
recommended for the service strategy because of rounding. 

(a) Miscellaneous program services include such activities as assessment, 
job-readiness preparation, vocational exploration, job shadowing, and tryout employment. 
In this table, miscellaneous program services also include work experience, which is 
identified as a separate program service in other reports on the National JTPA Study. 
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and 30 percent of male youths. Staff recommended the OJT/JSA service strategy for 49 percent 
of adult men and 33 percent of male youths compared with 35 percent of adult women and 23 
percent of female youths. 5 

Table 42 presents JTPA enrollment rates for treatment group members recommended for 
each service strategy, by target group. In the 18-month treatment group sample, 64 percent of 
the treatment group were enrolled in JTPA at some point in the 18-month follow-up period 
covered by this report, as shown in the bottom row of the column marked "All Target Groups." 
The rows show enrollment rates for each of the three service strategies, with classroom training 
having the highest rate (72 percent) and OJT/JSA the lowest rate (57 percent) for all target 
groups. 6 Within each service strategy, the variation in enrollment rates across the target groups 
is relatively small. Aggregating across all service strategies, adult men were slightly less likely to 
enroll than were the other target groups. 

Moving beyond this summary measure of JTPA enrollment, it is useful to describe the key 
services received by individuals in each sendee strategy. This can be done in two ways: (1) by 
calculating the proportion of all treatment group members enrolled in specific services and (2) 
by focusing on those who enrolled and calculating the proportion of enrollees who received 
specific services. Both measures are useful and are presented in the following discussion. The 
first measure is useful in characterizing the overall "strength" of the JTPA services received by the 
entire treatment group. In the impact analysis, the primary impact estimate is calculated by 
comparing the experiences of treatment group members with control group members, so this 
measure is important in assessing the additional services received by the treatment group 
compared with the services received by the control group. 7 The second measure — services 
received by those who enroll — is the same as that used in national reports on JTPA services. 
This second measure, because it involves a shift to the smaller base of enrollees, produces higher 
rates of receipt for individual services. 

Table 43 summarizes the JTPA enrollment data for the treatment group members by 
presenting information on the two key services, by target grc jp and service strategy. The top 
panel presents data for all members of the treatment group; e.g., 61 percent of all adult male 



5 Members of the nonexperimental analysis team of the National JTPA Study are seeking ways to 
model statistically the process of assessment and service recommendation in four of the 16 study SDAs. 
See Bloom et al., 1990, Chapter 6, for a discussion of the plans for this research. 

6 A later section of this chapter discusses possible reasons for this pattern. 

7 The 18-month impact report presents information on the differences between the treatment and control 
groups in service receipt as part oi its analysis of program impacts. 

69 



TABLE 4.2 



ENROLLMENT RATES FOR TREATMENT GROUP MEMBERS, 
BY SERVICE STRATEGY AND TARGET GROUP 



Service Strategy 


All Target 
Groups 

(%) 


Adult 
Men 

(%) 


Adult 
Women 

(%) 


A M t 

Male 
Youths 
(%) 


Female 
Youths 
(*) 


Classroom Training 


72.4 


71.2 


72.8 


74.8 


71.5 


OJT/JSA 


56.5 


56.6 


55.4 


58.5 


57.5 


Other Services 


62.3 


58.9 


62.4 


67.7 


63.1 


All Strategies 


63.8 


60.8 


64.6 


66.8 


65.5 



SOURCE: MDRC calculations from Background Information Form responses and program 
enrollment data from the 16 SDAs. 

NOTE: Calculations for this table ar: based on data for all treatment group members in the 
18-month study sample. 
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TABLE 4.3 



KEY SERVICES FOR TREATMENT GROUP MEMBERS AND ENROLLEES, 
BY TARGET GROUP AND SERVICE STRATEGY 



Sample and 

Target Group 



Classroom 
Training 



OJT/JSA 



Other 
Services 



Treatment 
Group Members 



Adult Men 



61 % enrolled in 
CT-OS or BE 



49% enrolled in 
OJT or JSA 



52% enrolled in 

JSA or misc. services (a) 



Adult Women 



65% enrolled in 
CT-OS or BE 



49% enrolled in 
OJT or JSA 



52% enrolled in 

JSA or misc. services (a) 



Male Youths 



60% enrolled in 
CT-OS or BE 



49% enrolled in 
OJT or JSA 



56% enrolled in 

BE or misc. services (a; 



Female Youths 



62% enrolled in 
CT-OS or BE 



49% enrolled in 
OJT or JSA 



50% enrolled in 

BE or misc. services (a) 



Enrollees 
Adult Men 

Adult Women 

Male Youths 

Female Youths 



86% enrolled in 
CT-OS or BE 

89% enrolled in 
CT-OS or BE 

80% enrolled in 
CT-OS or BE 

86% enrolled in 
CT-OS or BE 



87% enrolled in 
OJT or JSA 

88% enrolled in 
OJT or JSA 

85% enrolled in 
OJT or JSA 

85% enrolled in 
OJT or JSA 



89% enrolled in 

JSA or misc. services (a) 

82% enrolled in 

JSA or misc. services (a) 

83 % enrolled in 

BE or misc. services (a) 

80% enrolled in 

BE or mi&x\ services (a) 



SOURCE: MDRC calculations from Background Information Form responses and from 
program enrollment and participation data from the 16 SDAs. 

NOTES: Calculations for this table are based on data for all treatment group members and 
enrollees in the 18-month study sample. 

(a) Miscellaneous program services include such activities as assessment, job-readiness 
preparation, vocational exploration, job shadowing, and tryout employment. In this table, 
miscellaneous program services also include work experience, which is identified as 
a separate program service in other reports on the National JTPA Study. 



treatment group members recommended for the classroom training service strategy were enrolled 
in CT-OS, basic education (BE), or both. The bottom panel presents data for those who enrolled 
in JTPA, Le., the percentage of enrollees receiving the two key services. 

Both measures show a clear emphasis for each service strategy, as envisioned in the research 
design. The lower panel on enrollees shows the following emphases: 

• Classroom trainings Most enrollees were active in classroom-based services of 
some type. Eighty-six percent of adult male enrollees, 89 percent of adult 
female enrollees, 80 percent of male youth enrollees, and 86 percent of female 
youth enrollees were active in CT-OS, basic education, or both; 

• OJT/JSA: Most enrollees were active in OJT or job search assistance. Eighty- 
seven percent of adult male enrollees, 88 percent of adult female enrollees, 85 
percent of male youth enrollees, and 85 percent of female youth enrollees were 
active in OJT, job search assistance (JSA), or both; and 

• Other services: The predominant types of services differed between adults and 
youths. Eighty-nine percent of adult male enrollees and 82 percent of adult 
female enrollees were active in job search assistance or miscellaneous services, 
while 83 percent of male youth enrollees and 80 percent of female youth 
enrollees were active in basic education or miscellaneous services. 

Table 4.4 provides more detail on the nature of the JTPA services in which treatment 
group members in each service strategy enrolled 8 Like Table 43, this table includes data for 
all treatment group members in the top panel and for enrollees in the bottom panel. It shows 
the percentages of those enrolled in JTPA who received various JTPA services. Since enrollees 
could receive more than one of the listed services, the percentages for enrollment in the 
individual services in the top panel add to more than the overall enrollment rate in JTPA and the 
percentages in the bottom panel add to more than 100 percent The first column presents 
information for all strategies combined, and the remaining columns list the three service strategies. 

The second column of the bottom panel (enrollees in the classroom training strategy) shows 
that 78 percent of enrollees received CT-OS and 18 percent received basic education. Thus, local 
staff were relatively successful in getting treatment group members in this service strategy into 
their recommended primary service of CT-OS. In addition, 5 percent received OJT, 27 percent 
received job search assistance, and 19 percent received miscellaneous services. 

In the OJT/JSA service strategy, 50 percent of enrollees received OJT and 51 percent 
received job search assistance. Thus, the vast majority of enrollees received some type of service 



'Detailed tables for each of the target groups are included in Appendix B. 
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TABLE 4.4 



SERVICE RECEIPT RATES FOR TREATMENT GROUP MEMBERS AND ENROLLEES, 

BY SERVICE STRATEGY 



Sample and 

Program Service Received 



All 
Strategies 



Classroom 
Training 



OJT/JSA 



Other 
Services 



Treatment Group Members 

Ever Received (%) 
Classroom training in 

occupational skills (CT-OS) 
On-the-job training (OJT) 
Job search assistance (JSA) 
Basic education (BE) 
Miscellaneous services (a) 

Sample Size 



23.9 


56.2 


3.3 


9.4 


13.1 


3.8 


28.0 


4.7 


23.0 


19.5 


28.9 


19.7 


10.0 


12.9 


3.1 


15.7 


16.6 


13.4 


8.6 


31.9 


11,474 


4,123 


4,287 


3,064 



Enrollees 



Ever Received (%) 
Classroom training in 

occupational skills (CT-OS) 
On-the-job training (OJT) 
Job search assistance (JSA) 
Basic education (BE) 
Miscellaneous services (a) 

Sample Size 



37.5 


77.6 


5.8 


15.1 


20.5 


5.2 


49.6 


7.5 


36.1 


26.9 


51.1 


31.6 


15.7 


17.8 


5.5 


25.2 


26.0 


18.6 


15.2 


51.1 


7,316 


2,986 


2,421 


1,909 



SOURCE: MDRC calculations from Background Information Form responses and from 
program enrollment and participation data from the 16 SDAs. 

NOTES: Calculations for this table are based on data for all treatment group members 
and enrollees in the 18-month study sample. 

In each panel, the total service receipt rate in each service strategy may be over 100.0 
percent because some treatment group members and enrollees received more than one service. 

Tests of statistical significance were not performed. 

(a) Miscellaneous program services include such activities as assessment, job-readiness 
preparation, vocational exploration, job shadowing, and tryout employment. In this table, 
miscellaneous program services also include work experience, which is identified as 
a separate program service in other reports on the National JTPA Study. 
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designed to place them immediately in an unsubsidized job or a job providing subsidized training. 
In addition, 15 percent received miscellaneous services. 

As planned, the other services strategy comprises a mixture of services. This reflects the 
fact that very different types of applicants could be referred to this service strategy. It includes 
the most job-ready applicants, who were recommended only for job search assistance, and those 
for whom pretraining services such as basic education or preemployment skills and exposure to 
the world of work (included within miscellaneous services in Tables 43 and 4.4) were appropriate. 
Consequently, the services received varied between adults and youths, with job search assistance, 
basic education, and miscellaneous services all being important 

The operational reality of tasks involved in arranging different types of JTPA services helps 
explain the differing enrollment rates and service receipt across the three service strategies. For 
classroom training, JTPA staff must identify a service provider (Lc, a community college, technical 
institute, proprietary school, or community-based organization) that offers training the client wants 
and that will accept someone with the client's qualifications. In many SDAs, especially those that 
include urban areas, there are several agencies whose mission is to provide training. 9 

While making a match between a client and a classroom training provider is not always 
easy, the JTPA agency and the service provider have similar missions (providing training) and 
may well see each other as continuing partners. JTPA may be a major funder of training in a 
community, especially at community-based organizations and other agencies without major state 
government grants-in-aid. JTPA staff report that they often know the details of entrance 
requirements for different types of training and use this in developing recommendations for 
individual clients. In some jurisdictions, training agencies may even accept the test results and 
assessment made by JTPA staff. 10 These factors suggest that a relatively high proportion of the 
classroom training program group will be placed with a training agency and enrolled in JTPA. 

The process for developing an OJT position differs substantially and in ways likely to lead 
to lower rates of enrollment - both in OJT and in JTPA overall - for the OJT/JSA service 
strategy. Private employers' main goal is producing goods or services, not subsidized training. 
They must be induced to participate as employers and trainers through a combination of financial 



9 Among the SDAs in the study, rural sites such as Butte, Montana, and Northwest Minnesota worked 
with as few as three classroom training service providers for adults and out-of-school youths, while more 
urban sites such as Fort Wayne, Indiana, typically contracted with 20 or more agencies for training. 

10 Service providers may be hesitant to do this when they are paid through a performance-based contract 
with a substantial proportion of funds tied to successful completion of training and placement in a job. 
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incentives and assurances that the administrative costs will not overwhelm the subsidy they receive. 
Even in small programs, local staff have to develop OJT positions with many different employers. 
With minority clients, staff may face employer discrimination, which can be subtle and hard to 
document, and employers considering offering an OJT position typically retain the hiring authority. 
Finally, clients are likely to be actively seeking employment (as those slated for the classroom 
training strategy often are not), either on their own or through other program services such as 
job search assistance. If a client does find an unsubsidized job, it could result in no JTPA 
enrollment (if the placement was unrelated to the program) or to an enrollment in job search and 
a placement 

These operational realities also affect the experiences of those members of the treatment 
group who were nevcv enrolled in JTPA. Table 4.5 is based on the special survey of the 
nonenrolled treatment group members referred to in Table 3.2. It shows that in the classroom 
training strategy, most of those who were never enrolled in JIPA either lost contact with the 
program (28 percent), were no longer interested in JTPA (19 percent), or did not receive any 
substantial services from local staff (28 percent). In this service strategy, for the remaining 23 
percent of the nonenrolled, local staff worked unsuccessfully to arrange a service and enroll the 
client 

The situation for OJT/JSA is in sharp contrast Staff worked unsuccessfully with 68 percent 
of the nonenrolled to arrange a service and enroll the client. Fifty-seven percent of this sample 
of f he nonenrolled were referred to employers for a potential OJT position, while 23 percent 
received other help finding a job. 

The data in this table provide supporting evidence that arranging placement in a service 
that results in enrollment in JTPA is easiest in the classroom training service strategy. The JTPA 
enrollment rate is highest for those recommended for this strategy, and the rate at which staff 
work with clients without producing an enrollment is lowest (23 percent rate for classroom 
training versus 68 percent for OJT/JSA and 49 percent for other services). 

To summarize, nonenrolled post-random-assignment involvement in JTPA was most common 
among those in the OJT/JSA service strategy and least common among those in the classroom 
training strategy. Since JTPA enrollment rates follow an opposite pattern, when nonenrolled data 
are combined with enrolled data, differences in involvement with JTPA among the service 
strategies diminish. Those in the OJT/JSA strategy were slightly more likely (86 percent) to have 
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TABLE 4.5 



PERCENTAGE DISTRIBUTIONS OF POST-RANDOM ASSIGNMENT 
ACTIVITY IN JTPA OF TREATMENT GROUP MEMBERS WHO DID NOT ENROLL, 

BY SERVICE STRATEGY 





Classroom 




Other 




Training 


OJT/JSA 


Services 




(%) 


(%) 


<*) 


No Further Contact 


28.1 


5.4 


ZU.O 


runner ion tec i» tsui woe cugiDie 


1 ft 


0.7 


2.0 


No Longer Interested 


19.3 


10.1 


7.8 


TnterMttt) Rut Made Cmtact OnlV 








iwi Received No Services 


28.1 


16.2 


21.6 


Interested and Received Services) (a) 


22.8 


67.6 


49.0 


Received further assessment 








and counseling 


10.5 


10.8 


11.8 


Referred to classroom 








training providers) 


10.5 


2.7 


5.9 


Received support service(s) 


1.8 


3.4 


1.0 


Referred to employees) for 








possible on-the-job training 


3.5 


56.8 


24.5 


Participated in job club or 








received job search assistance 


0.0 


23.0 


26.5 


Total 


100.0 


100.0 


100.0 


Sample Size 


57 


148 


102 



SOURCE: Information collected by MDRC site representatives during the National 
JTPA Study. 



NOTES: Calculations for this table are based on data for a random sample of 
307 treatment group members in the 18-month study sample who did not enroll in 
JTPA. 

Distributions may not total 100.0 percent because of rounding. 

(a) When totaled, the subcategory percentages in each service strategy are over 
22.8, 67.6, and 49.0 percent, respectively, because some nonenrollees received more than 
one sen 'ice. 
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had some post-random-assignment involvement with JTPA than people in the other strategies (81 
percent for other services and 79 percent for classroom training). 11 

To sum up, three main findings emerge from this chapter First, die chosen point of 
random assignment produced estimates of the overall impact of JTPA for the entire sample and 
target groups and subgroups, as well as impacts for individuals recommended for three service 
strategies. Second, the research design allows calculation of estimates in two ways: impacts per 
assignee (based on a comparison of the treatment and control groups), which can be converted 
to impacts per enrollee, though the assumptions underlying this conversion (while they appear 
reasonable) cannot be tested Third, the service strategies analyzed in the implementation and 
impact research did have clear programmatic emphases. Hie focus of the classroom training 
strategy was as expected, while OJT/JSA involved job placement (through subsidized OJT or job 
search). The other services strategy emphasized job search assistance and miscellaneous services 
for adults, and basic education and miscellaneous services for youths. 



u The sample for this special study was too small to allow simultaneous disaggregation by target group 
and service strategy. 
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CHAPTER 5 
SERVICES RECEIVED BY ADULT FNROI J.KES 



This chapter describes the distribution of JTPA services among adult members of the 18- 
month treatment group sample who were officially enrolled in JTPA. 1 The analysis shifts away 
from the focus of the previous chapters on artifacts of the research design such as distinctions 
between all treatment group members and enrollees and the concept of "recommended service 
strategies/ For the analysis in this chapter, adult enrollees have been grouped according to the 
types of services they actually received rather than the service strategies for which they were 
recommended at assessment and random assignment Within this framework, the chapter analyzes 
service receipt rates and the length of enrollment for the adult target groups (men and women) 
and other key subgroups of adult enrollees. 

Readers of the 18-month impact report can use the information in Chapters 5 and 6 as a 
context for understanding a key determinant of program impacts: the types and duration of 
services that enrolled treatment group members received. 2 The chapters also provide a 
perspective on how JTPA resources were distributed among key demographic groups that the 
SDAs tried to target for services. 

This chapter (and Chapter 6, which focuses on out-of-school youths) organizes the analysis 
of services received and length of enrollment around five mutually exclusive categories. The five 
categories divide the sample into two groups of enrollees who received some form of occupational 
training (either in the classroom or on the job) and Juree groups who received other services 
without occupational training. The categories are: (1) classroom training in occupational skills 
(CT-OS) only or CT-OS in combination with on-the-job training (OJT), basic education, job 
search assistance, or miscellaneous services (referred to in the tables and text as CT-OS); (2) OJT 
only or OJT in combination with basic education, job search assistance, or miscellaneous services 



l As noted in earlier chapters, treatment group members were identified as 'officially enrolled in JTPA" 
if the local SDA staff established an enrollment record on their JTPA management inforroation system 
(MIS) during the 18-month follow-up period. 

2 See Bloom et al., 1993, for an analysis of another key determinant of program impacts: the extent 
to which control group members received equivalent services from sources other than JTPA 
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(referred to in the tables and text as OJT); 3 (3) basic education only, without occupational 
training, or basic education in combination with job search assistance or miscellaneous services 
(referred to in the tables as non-training education and in the art as basic education without 
training); 4 (4) job search assistance only; and (5) miscellaneous sendees only or miscellaneous 
services in combination with job search assistance (referred to in the tables and text as 
miscellaneous non-training services). 

These five service categories provide a framework for examining service receipt rates for 
JTPA enrollees and for analyzing the determinants of a key facet of service intensity: length of 
enrollment Following is an overview of key findings in this chapter. 

• A majority of the enrolled adults received some form of occupational training 
through either CT-OS or OJT. Thirty-eight percent of the adult enrollees 
received CT-OS and an additional 21 percent received OJT* Nineteen percent 
of the adult enrollees received job search assistance only, and the remainder 
received either basic education without training or other non-training services. 

• Adult enrollees with two or three of the employment barriers analyzed (Ugh 
school dropout, limited work experience, and cash public assistance recipient) 
were less likely to receive OJT and Job search assistance only and more likely 
to receive basic education without training than those with no employment 
barriers. Adults with limited work experience or who were receiving public 
assistance were more likely to receive classroom-based services such as CT-OS 
and basic education without training and less likely to receive employment-based 
services such as OJT and job search assistance only receipt than adults without 
these employment barriers. Adults with no high school diploma or GED were 
less likely to receive CT-OS and more likely to receive basic education without 
training than high school graduates or GED recipients. 

• Women tended to receive classroom-based services while men tended to receive 
employment-based services. Employment barriers accounted for much of the 
gender difference in OJT receipt, but very little of the difference in CT-OS 
receipt Overall, 56 percent of the adult women — compared with 34 percent 
of the men — received either CT-OS or basic education without training. By 
contrast, 49 percent of the adult men — compared with 33 percent of the 
women — received OJT or job search assistance only. 



3 Note that the 13 percent of the adult enrollees who received both CT-OS and OJT are grouped only 
in the CT-OS service category. This is because CT-OS was often the primary service, with an OJT used 
as a temporary link to an unsubsidized job. 

4 Basic education includes a range of education services: adult basic education (commonly referred to 
as ABE), GED preparation, English as a second language classes (commonly referred to as ESL), and 
community college courses. These are all grouped under the term "basic education" in order to differentiate 
them from CT-OS. 
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• Black and Hispfmk enrollees were more likely to receive CT-OS and less 
likely to receive OJT than white enrollees. Employment barriers accounted for 
some, but not all, of the ethnic differences in service receipt Fifty-one percent 
of the black enrollees and 39 percent of the Hispanic enrollees received CT- 
OS compared with 31 percent of the white enrollees. By contrast, 26 percent 
of the white enrollees received OJT compared with 16 percent of the black 
enrollees and 11 percent of the Hispanic enrollees. 

• The median length of JTPA enrollment for adults in the five service categories 
ranged from almost five months to only one month. Overall, the median 
length of enrollment for adult enrollees was 32 months. The median length of 
enrollment was longest at 4.8 months for those who received CT-OS and 
shortest at one month for those who received job search assistance only. The 
median length of enrollment for adults who received OJT was JL5 months. 

These general findings art discussed in greater detail in the following sections of the 
chapter. 

I. Service Receipt in the Five Service Categories 

After describing the overall distribution of services among enrolled adults, this section 
presents three levels of analysis of the types of services received by adults. It examines the 
services received by each of the two adult target groups (men and women), analyzes differences 
in service receipt rates among selected subgroups of adults, and, finally, examines whether service 
receipt differences among adults with various combinations of employment barriers account for 
gender and ethnic differences in service receipt 

The top row in Table 5.1 lists the service receipt rates for adult enrollees in each of the 
five service categories. Overall, 59 percent of the adults received some type of occupational 
training - either in the classroom or on the job. This number was calculated by adding the 
percentages of all adults who ever received CT-OS (38 percent) and of all adults who ever 
received OJT (21 percent). The table also shows that 19 percent of the adults received job 
search assistance as their only service, 8 percent received basic education without training, and the 
remaining 14 percent received miscellaneous non-training services. 

The remainder of Table 5.1 lists selected subgroups of adult enrollees and the percentage 
of each subgroup who received each category of services. Statistical tests were performed to 
determine whether there were significant differences in service receipt rates between the first 
subgroup listed under each characteristic and each of the other subgroups. For example, among 
the subgroups defined by ethnicity, the table indicates that there was a statistically significant 
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TABLE 5.1 



SERVICE RECEIPT RATES FOR ADULT ENROLLEES, 
BY SELECTED BASELINE CHARACTERISTICS 



Characteristic 



Sample CT-OS (a) 



Non- 
Training JSA 
OJT (b) Education (c) Only 



Misc. 
Non-Training 
Services (d) 



and Subgroup 


olZC 


\*>) 




{To) 




{70) 




{To) 




{70) 


All Adult Enrollees 


5,169 


38.0 




20.5 




7.9 




19.2 




14.0 


Target Group 






















Adult Men 

AUUll ItJlvU 


2 286 


26 3 




24.3 




7.2 




25 0 




17.2 


/vuuii women 




Al 9 

H / .Z 




ift i 

10.1 


*** 


ft A 


* 


Id 7 


+++ 


1 1 f. +++ 

1 1 .o 


Age 






















2>29 


2,305 


39.1 




22.3 




8.2 




16.4 




14.0 


30-44 


2,239 


39.2 




19.2 


*** 


8.2 




19.5 


+++ 


13.9 


A C am/4 nmt^w 

*tj ■do over 


A9^ 
OZj 


90 £ 

zy.o 


*** 


91 1 




J.XJ 


++ 


9R R 


*** 




Ethnicity 






















White, non-Hispanic 


2,941 


30.6 




25.9 




9.1 




20.0 




14.5 


Black, non-Hispanic 


1 ill 

1,432 


50.5 




15.6 


♦ ♦ ♦ 


2.7 




19.7 




11C AAA 

11.5 *** 


Hispanic 


589 


38.7 




11.0 




15.8 




16.3 




18.2 ** 


Other 


207 


55.1 




13.0 




3.9 




14.0 




14.0 


Education 






















High school diploma or GED 


3,491 


41.3 




20.8 




4.0 




20.3 




13.6 


No hish school diploma or GED 


1,342 


29.4 


++* 


21.7 




17.8 


+*+ 


16.6 


*** 


14.5 


\JL//"»rV" pwyii jjim^ji 






















vtOikco ij> wccKb or mure 






















in 12 months before assignment 


2,428 


34.4 




23.7 




6.0 




21.9 




14.0 


Worked fewer than 13 weeks 






















in iz moDUis oerore assignment 


9 17ft 
Z, 1 / o 


AO 9 
4Z.Z 


*** 


17 5 

1 / . D 




Q 6 


*** 


16 1 

1U. 1 


*** 


14 6 


Public Assistance Receipt 






















Not receiving AFDC or other cash 






















assistance at assignment 


3,287 


34.8 




23.8 




5.1 




22.2 




14.1 


Receiving AFDC or other cash 






















assistance at assignment 


1,405 


47 J 


*** 


14.0 


*** 


14.7 


*** 


11.5 


*** 


12.7 


Barriers to Employment (e) 






















None 


1,828 


35.6 




24.1 




2.6 




23.5 




14.2 


1 


1,952 


37.5 




21.6 


* 


6.7 


*** 


20.5 


** 


13,7 


2 


1,092 


42.4 




16.8 


+ ++ 


13.6 


*** 


13.2 


*** 


14.0 


3 


263 


403 




10.3 


* + + 


28.5 


+++ 


6.1 


*** 


14.8 



(continued) 
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TABLE 5.1 (continued) 



Characteristic 



Non- Misc. 
Training JSA Non-Training 
Sample CT-OS (a) OJT (b) Education (c) Only Services (d) 



and Subgroup 


Size 




(%) 




<*) 




<%) 




<*) 


atuv rustoiy 




















Never received AFDC 


3,461 


32.7 


23.2 




6.4 




22.8 




14.8 


Received AFDC less than 




















2 years (f) 


837 


44.1 *** 


18.6 


*** 


10.4 


*** 


14.3 


*** 


12.5 * 


Received AFDC 2 years 




















or more (f) 


799 


54.1 *** 


13.6 


*** 


11.1 


*** 


9.0 


*** 


12.1 * 


Household Composition 




















Spouse present 


1,410 


31.3 


23.4 




9.4 




22.5 




13.4 


No spouse present, child present 


1,558 


49.5 *** 


18.0 


*** 


8.2 




12.6 


*** 


11.6 


No spouse present, no child 




















present 


1,633 


31.2 


22.7 




6.1 


*** 


23.1 




16.9 *** 



SOURCE: MDRC calculations from Background Information Form responses and from program enrollment 
and participation data from the 16 SDAs. 

NOTES: Calculations for this table are based on data for all adult enrollees in the 18-month study sample. 
The total sample size may vary among characteristics because of missing data for some enrollees. 

The program service categories in this table are mutually exclusive; see notes a, b, c, and d below. 

For each characteristic, within each program service category, a chi-square test was applied to the 
difference in service receipt rates between the first subgroup listed and, separately, each of the remaining subgroups. 
Statistical significance levels are indicated as * « 10 percent; ** « 5 percent; *** « 1 percent. 

(a) The CT-OS service category includes enrollees who received CT-OS only, or CT-OS in 
combination with OJT, basic education, job search assistance, or miscellaneous program services. 

(b) The OJT service category includes enrollees who received OJT only, or OJT in combination 
with basic education, job search assistance, or miscellaneous program services. 

(c) The non-training education service category includes enrollees who received basic education 
only, or basic education in combination with job search assistance or miscellaneous program services. 

(d) The miscellaneous non-training services category includes enrollees who received miscellane- 
ous program services only, or miscellaneous program services in combination with job search assistance. 
Miscellaneous program services include such activities as assessment, job-readiness preparation, vocational 
exploration, job shadowing, and try out employment. In this table, miscellaneous program services also 
include work experience, which is identified as a separate program service in other reports on the National 
JTPA Study. 

(e) The barriers to employment include: not having a high school diploma or GED at the time of 
random assignment; having worked fewer than 13 weeks during the 12 months prior to random assignment; 
and receiving AFDC or other cash assistance at the time of random assignment. 

(f) The AFDC receipt may not have been continuous. 
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difference between the percentage of white adults who received CT-OS (31 percent) and the 
percentage of black adults who received CT-OS (51 percent). Hie table also indicates that there 
was a statistically significant difference between the percentage of white adults who received CT- 
OS and the percentage of Hispanic adults who received CT-OS (39 percent). 5 

A. Service Receipt Rates for Each of the Adult Target Groups 

Table 5.1 shows several contrasts between men and women in the types of services they 
received. Overall, a total of 65 percent of the women received either CT-OS or OJT compared 
with only 51 percent of the men. The table also highlights the sharp contrast in the mode of 
occupational training (classroom versus on-the-job) that men and women received. Specifically, 
47 percent of the women received CT-OS compared with only 26 percent of the men, and 18 
percent of the women received OJT compared with 24 percent of the men. 

More generally, women tended to receive either type of classroom-based service - CT-OS 
or basic education without training - while men tended to receive either type of employment- 
based service - OJT or job search assistance only. Specifically, a total of 56 percent of the 
women received either CT-OS or basic education without training compared with 34 percent of 
the men. Conversely, a total of 49 percent of the men received OJT or job search assistance only 
compared with 33 percent of the women. 

Finally, men and women differed in the extent to which they received multiple services. For 
example, 30 percent of the women received two or more services compared with only 21 percent 
of the men. This difference reflects the fact that women were more likely than men to combine 
job search assistance and miscellaneous services with their classroom-based services. 6 Further 
analysis of the extent to which differences in service receipt rates associated with employment 
barriers account for target group differences in service receipt rates is discussed later in the 
chapter. 



Statistical tests to determine whether there were significant differences in service receipt rates between 
the second and third subgroups listed under each characteristic were not performed. In this example, there 
was no statistical test of the difference between the percentage of black enrollees who received CT-OS and 
the percentage of Hispanic enrollees who received CT-OS. Unless otherwise noted, discussion of service 
receipt differences among subgroups focuses on those that are statistically significant at the 10 percent level 
or lower. 

6 These percentages are based on MDRC calculations from enrollment and participation data from the 
16 SDAs in the National JTPA Study. 

83 



B. Service Receipt Rates for Subgroups of Adult Eorollces 

This section discusses the distribution of JTPA services among the subgroups of adult 
enrollees listed in Table 5.1. Although many of the subgroup differences in service receipt are 
similar for men and women, any especially large differences are indicated in the text, and 
Appendix B provides separate subgroup tables for adult men and women. 

1. Age subgroups. Table 5.1 indicates that the largest differences in service receipt 
among subgroups defined by age at random assignment are those between the oldest and youngest 
adult enrollees. Adults aged 45 and over were less likely than those aged 22-29 to receive CT- 
OS or ba*»ic education without training and more likely to receive job search assistance only. This 
suggests that age may serve, in part, as an indication of job-readiness or previous employment 
experience that orients older workers toward unsubsidized jobs rather than classroom-based 
services. 

2. Ethnic subgroups. Table 5.1 shows dramatic differences among white and black 
adults in the percentage who received CT-OS and OJT. In total, black adults were more likely 
to receive occupational training - either in the classroom or on the job - than white adults; 66 
percent of the black adults received CT-OS or OJT compared with 57 percent of the white 
adults. 7 However, white and black adults differed markedly in the mode of training they received. 
Specifically, 51 percent of the black adults received CT-OS compared with only 31 percent of the 
white adults, and only 16 percent of the black adults received OJT compared with 26 percent of 
the white adults. 8 

Two other black/white findings related to service receipt should be noted. First, black and 
white adults were equally likely to receive job search assistance a* their oniy service. Second, 
white adults were three times more likely than black adults to receive basic education without 
training. 9 This is surprising because 26 percent of blacks had no high school diploma or GED 
at the time of their JTPA application compared with 29 percent of whites. 



7 Note that differences in the total percentage who received any type of occupational training - 
combining the percentage who received CT-OS with the percentage who received OJT - were not tested 
for statistical significance. 

8 White/black differences in mode of occupational training (classroom versus on the job) that adults 
received were especially large among the men. For example, black men were twice as likely as white men 
to receive CT-OS, but they were half as likely to receive OJT (see Appendix B). 

9 Closer analysis also showed that white enrollees were much more likely than black enrollees to receive 
basic education in combination with CT-OS or OJT. 
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Table 5.1 also shows that Hispanic adults received different types of services than white 
adults. First, only 11 percent of the Hispanic adults received OJT compared with 26 percent of 
the white adults. In total, 50 percent of the Hispanic adults received occup&f! jnal training - 
either CT-OS or OJT — compared with 57 percent of the white adults. However, 16 percent of 
Hispanic adults received basic education without training compared with 9 percent of the white 
adults. The high rate of non-training education receipt among Hispanic adults may be due to the 
fact that ESL classes are included among these services. This suggests that language may have 
posed an important barrier to occupational training. 

Later in the chapter, the discussion returns to an analysis of the extent to which differences 
in service receipt rates associated with employment barriers account for differences in service 
receipt rates among ethnic groups. 10 

3. Employment barrier subgroups. This section focuses on the distribution of JTPA 
service among subgroups of adult enrollees defined by three employment barriers: those with no 
high school diploma or GED at the time of random assignment (referred to as high school 
dropout), those who worked few^r than 13 weeks in the year prior to random assignment 
(referred to as limited work ecperience), and those who were receiving AFDC or other cash 
assistance at the time of random assignment (referred to as cash public assistance recipient). 

Table 5.1 highlights several associations between these employment barriers and the types 
of services that adult enrollees were likely to receive. For example, high school dropouts were 
less likely to receive CT-OS than graduates or those who obtained a GED, but they were equally 
likely to receive OJT. In total, therefore, 51 percent of the high school dropouts received either 
CT-OS or OJT compared with 62 percent of those with a high school diploma or GED. Instead 
of receiving CT-OS, however, it appears that high school dropouts tended to receive basic 
education without training. Table 5.1 shows that dropouts were more than four times as likely 
to receive basic education as those with a diploma or GED. The relatively low percentage of 
high school dropouts who received CT-OS can be explained, in part, because some classroom 
training providers require a high school diploma or its equivalent for enrollment in their programs. 
However, very few of the SDAs in the study combined basic education and CT-OS for those who 
lacked that educational credential. 



10 It should be noted that these differences in service receipt rates do not appear to have translated into 
statistically significant differences in program impacts on earnings by ethnicity. See Bloom et al., 1993. 
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Not surprisingly, adults with limited work experience were less likely to receive OJT or job 
search assistance only than those who worked 13 weeks or more in the previous year. This may 
be due, in part, to the fact that those with limited work experience are likely to be less attractive 
^ to employers who may tend to select the most qualified people for their OJT positions or 
unsubsidized jobs. In total, however, 60 percent of those with limited work experience received 
some type of occupational training - either CT-OS or OJT - compared with 58 percent of 
those with more work experience. 11 This suggests that those with limited work experience were 
able to substitute occupational training in the classroom for the OJT they did not obtain or did 
not want 

Table 5.1 also shows that cash public assistance receipt was associated with higher rates of 
CT-OS receipt This is especially significant for women, who were much more likely than men 
to be receiving public assistance at the time of JTPA application. Those who were not receiving 
public assistance at JTPA application were almost twice as likely to receive job search assistance 
only as those who were. However, public assistance recipients were almost three times as likely 
to receive basic education without training as those who were not receiving AFDC or other cash 
assistance at the time of random assignment This may be due to the fact that approximately 75 
percent of the adult cash public assistance recipients also lacked a high school diploma or GED 
or had only limited work experience. 

Service receipt rates were also associated with the number of employment barriers that 
adults possessed. For example, adults with multiple employment barriers were less likely than 
those with no employment barriers to receive OJT or job search assistance only and were more 
likely to receive CT-OS or basic education without training. 

4. AFDC history and household composition subgroups . Finally, Table 5.1 shows that 
AFDC history and household composition were associated with service receipt rates. Like those 
who were receiving cash assistance at the time of random assignment, long-term AFDC recipients 
were also more likely to participate in classroom-based services and less likely to participate in 
employment-based services than adults who had never received AFDC. This suggests that those 
on public assistance may have been able to forgo earnings in order to invest in education and 



n This finding reflects the average for all adults, but not necessarily for each subgroup of adults. For 
example, men with limited work experience were less likely to receive occupational training - CT-OS or 
OJT - than men with more work experience. This is primarily because men with limited work experience 
were no more likely to receive CT-OS than men with moi e work experience. For a separate analysis of men 
and women, see Appendix B. 
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training. This is supported by the fact that half of the single parents - those most likely to have 
a history of AFDC receipt - received CT-OS. 

£ Further Discussion of Key Findiras 

Three sets of findings stand out from the analyses discussed in the previous sections: gender 
differences in service receipt, ethnic differences in service receipt, and the association between 
employment barriers and the types of services that adults received Recently, considerable 
attention has been focused on differences in the distribution of JTPA services among gender, 
ethnic, and job-readiness groups of JTPA participants. 12 This section addresses the question of 
whether, within the 16 study sites, differences in service receipt rates associated with employment 
barriers account for differences in the services received among gender and ethnic groups. 
Specifically, the analysis examines gender and ethnic differences in service receipt within 
subgroups of enrollees defined by the various combinations of the three employment barriers 
analyzed earlier (high school dropout, limited work experience, and cash public assistance 
recipient). This will permit a further investigation of the hypothesis that male and female adults 
(or black and white adults) with the same employment barriers were equally likely to receive a 
particular service. 

Because many of the black and Hispanic adults were concentrated in the larger urban SDAs 
and because black and Hispanic adults differed from white adults in characteristics other than 



12 One highly publicized st idy investigated disparities in the services provided to gender and ethnic 
groups of JTPA participants (see U.S. General Accounting Office, 1991). This study analyzed aggregate data 
from SDAs in 16 states and participant-level data in seven large-city SDAs. The report focused on its 
findings that, in general, women received classroom training more often than men. 

This report also focused on its findings that in 20 percent of the SDAs it studied, white participants 
were more likely to receive classroom training than minority group members; in 13 percent of the SDAs, 
white participants were more likely to receive OJTs; and in 18 percent of the SDAs, minority group 
members were more likely to receive job search assistance only. In the rest of the SDAs, services were 
either distributed equally among ethnic and gender groups or minority and female participants were more 
likely than white male participants to receive high-skill occupational training. 

A second study investigated the relationship between die level of participants' job-readiness and the 
types of services they received. (See U.S. General Accounting Office, 1989.) The investigators used four 
"employment barriers" (lacking recent work experience, being a high school dropout, receiving AFDC or 
general welfare, and being black or Hispanic) to identify groups of JTPA participants who were more or 
less job-ready. The report emphasized its finding that 60 percent of the "less job-ready" participants received 
occupational training (either classroom or OJT) compared with 72 percent of the more job-ready. In 
addition, approximately one third of the more job-ready participants received training in "high-skill 
occupations" compared with 16 percent of the less job-ready participants. The investigators also reported 
that 22 percent of the more job-ready participants received job search assistance only compared with 27 
percent of the less job-ready participants. 
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employment barriers, the analysis of ethnic differences in service receipt is extended further. 
Specifically, ethnic differences in service receipt were tested within a multivariate regression 
framework that adjusted for other differences in service receipt that may have been due to SDA 
practices and characteristics or to other individual characteristics such as age, household 
composition, and AFDC history. 

1. Further analysis of gender differences in service receipt The top panel of Table 5.2 
is taken directly from Table 5.1 and shows that adult women were more likely to receive CT-OS 
and less likely to receive OJT or job search assistance only than adult men. The contrast in 
service receipt rates between adult men and women can be traced, in part, to differences in 
services for which they were recommended at assessment (see Table 4.1). While 44 percent of 
the women were recommended for the classroom training, only 25 percent of the men were 
recommended for it Also, almost half of the men were recommended for OJT compared with 
only 35 percent of the women (see Table 4.1). 

The differences in service receipt among men and women may also be explained, in part, 
by differences in the types of employment barriers they possessed. For example, men were more 
likely to have recent work experience - an important hiring criterion for employers interested in 
selecting OJT recipients — than women (see Table 2.12). Adult women were more likely than 
adult men to be receiving public assistance, which may have provided the financial support that 
enabled them to forgo earnings in order to invest in education and training (see Table 2.12). 

In general, employment barriers accounted largely for target group differences in OJT 
receipt, but they did not account for target group differences in CT-OS receipt The remaining 
panels in Table 52 display service receipt rates for adult men and women within subgroups 
defined by each combination of the three employment barriers. The statistical significance levels 
indicate whether the percentage of men who received a particular category of services differed 
from the percentage of women who received that category of services. For example, the second 
panel indicates that among adults with no employment barriers, 43 percent of the women received 
CT-OS compared with 29 percent of the men. The statistical significance level indicates that this 
difference was statistically significant at the 1 percent level. 

Table 5.2 shows that among adults with the same employment barriers, adult women were 
more likely to receive CT-OS than adult men. However, women in all but two of the 
employment barrier subgroups were just as likely as their male counterparts to receive OJT. 
Table 52 also shows that men were more likely than women to receive job search assistance only 
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TABLE 5.2 



SERVICE RECEIPT RATES FOR ADULT ENROLLEES, 
BY EMPLOYMENT BARRIER SUBGROUP AND TARGET GROUP 



Employment Barrier Subgroup (a) 
and Target Group 



Sample 
Size 



CT-OS (b) 



Non- 
Training JSA 
OJT (c) Education (d) Only 
(%) (%) (%) 



Misc. 
Non-Training 
Services (e) 
(%) 



All Employment 
Barrier Subgroups 

Adult men 
Adult women 

No Employment Barriers 

Adult men 
Adult women 



2,286 
2,883 



949 
879 



26.3 24.3 
47.2 *** 18.1 *** 



28.7 
43.1 



*** 



25.5 
22.5 



7.2 
8.4 * 



2.0 
3.3 



25.0 17.2 
14.7 *** 11.6 *** 



26.6 
20.3 



*** 



17.3 
10.8 



*** 



1 Employment Barrier 

Worked fewer than 13 weeks 
in 12 months before assignment 

Adult men 

Adult women 



493 30.4 21.7 5.3 
511 48.7 *** 18.0 3.9 



24.5 18.1 
18.4 ** 11.0 *** 



No high school diploma or GED 
at assignment 

Adult men 

Adult women 



318 
235 



21.1 
35.7 



*** 



30.2 8.8 25.5 14.5 

23.0 * 13.2 15.7 *** 12.3 



Receiving AFDC or other cash 
assistance at assignment 

Adult men 

Adult women 



95 

300 



24.2 
53.0 



*** 



23.2 8.4 29.5 14.7 

17.0 5.7 13.0 *** 11.3 



2 Employment Barriers 

Worked fewer than 13 weeks 
in 12 months before assignment 
and no high school diploma or GED 
at assignment 

Adult men 
Adult women 



192 
153 



18.2 
35.3 



*** 



23.4 
26.8 



10.9 
14.4 



24.0 
12.4 



23.4 
11.1 



Worked fewer than 13 weeks 
in 12 months before assignment 
and receiving AFDC or other cash 
assistance at assignment 

Adult men 

Adult women 



99 
467 



31.3 
63.0 



»»» 



22.2 
10.1 



*** 



7.1 
7.9 



24.2 
6.6 



15.2 
12.4 

(continued) 
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TABLE 5.2 (continued) 



Non- Misc. 

Training JSA Non-Training 

Employment Barrier Subgroup (a) Sample CT-OS (b) OJT (c) Education (d) Only Services (e) 

and Target Group Size (%) (%) (%) (%) (*) 

No high school diploma or GED 
at assignment and receiving 
AFDC or other cash assistance 
at assignment 

Adult men 

Adult women 

All 3 Emp lftyipftnt Barriers 

Adult men 62 19.4 11.3 37.1 9.7 22.6 

Adult women 201 46.8 *** 10.0 25.9 * 5.0 12.4 * 



SOURCE: MDRC calculations from Background Information Form responses and from program 
enrollment and participation data from the 16 SDAs. 

NOTES: Calculations for this table are based on data for all adult enrollees in the 18-month study sample. 

The program service categories in this table are mutually exclusive; see notes b, c, d, and e below. 

For each employment barrier subgroup, within each program service category, a chi-square test was 
applied to the difference in receipt rates between adult men and adult women. Statistical significance 
levels are indicated as * ■ 10 percent; ** - 5 percent; *** - 1 percent. 

(a) The barriers to employment include: not having a high school diploma or GED at the time of 
random assignment; having worked fewer than 13 weeks during the 12 months prior to random assignment; 
and receiving AFDC or other cash assistance at the time of random assignment. 

(b) The CT-OS service category includes enrollees who received CT-OS only, or CT-OS in 
combination with OJT, basic education, job search assistance, or miscellaneous program services. 

(c) The OJT service category includes enrollees who received OJT only, or OJT in combination 
with basic education, job search assistance, or miscellaneous program services. 

(d) The non-training education service category includes enrollees who received basic education 
only, or basic education in combination with job search assistance or miscellaneous program services. 

(e) The miscellaneous non-training services category includes enrollees who received miscellane- 
ous program services only, or miscellaneous program services in combination with job search assistance. 
Miscellaneous program services include such activities as assessment, job-readiness preparation, vocational 
exploration, job shadowing, and tryout employment. In this table, miscellaneous program services also 
include work experience, which is identified as a separate program service in other reports on the National 
JTPA Study. 



142 



65 13.8 18.5 46.2 16.9 4.6 
116 34.5 *** 13.8 27.6 ** 11.2 12.9 
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in all the employment barrier subgroups. Note, however, that gender differences in job search 
assistance only receipt are not statistically significant in the last two employment barrier 
subgroups. 

There are several possible explanations for the remaining difference in CT-OS service, 
receipt The service preference of men and women may differ, or men and women as a group 
may differ on other characteristics not included in the analysis that affect their appropriateness 
for CT-OS or the likelihood that service providers will accept them for training. Finally, SDA 
and service provider assessment and referral practices may 'treat similar men and women 
differently. The information available for this report could not be used to isolate the importance 
of these possible causes of the CT-OS service difference. 

2. Further analysis of ethnic differences in service receipt The top panel of Table 53 
is taken directly from Table 5.1 and shows that black adults were more likely to receive CT-OS 
and less likely to receive OJT and basic education without training than white adults. Like the 
gender differences in service receipt rates discussed above, white/black differences in service 
receipt rates can be traced to the types of services that were recommended at assessment for 
white and black adults. Black adults were much more likely than white adults to be 
recommended for CT-OS and much less likely to be recommended for OJT. Another explanation 
for the differences in service receipt rates among ethnic groups may be that ethnicity is associated 
with employment barriers. Recall that black adults (particularly women) were more likely to 
possess multiple employment barriers than white adults (see Table 2.14). Also recall that, in 
general, adults with multiple employment barriers were the least likely to receive employment- 
based services such as OJT and job search assistance only and the most likely to receive 
classroom-based services such as CT-OS and basic education. 

In general, employment barriers did not account consistently for the differences in service 
receipt rates among black and white adults. The remaining panels in Table 5.3 display differences 
in service receipt rates among white, black, and Hispanic adults within subgroups defined by each 
combination of the three employment barriers. 13 The statistical significance levels indicate 
whether the percentage of white adults who received a particular category of services differed 
from the percentage of either black or Hispanic adults who received that category of services. 
For example, the second panel shows that among adults with no employment barriers, 30 percent 



13 Adults from "other" ethnic groups were not included in the analysis because of small sample sizes. 
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TABLE 5.3 



SERVICE RECEIPT RATES FOR ADULT ENROLLEES, 
BY EMPLOYMENT BARRIER SUBGROUP AND ETHNICITY 



Employment Barrier Subgroup (a) Sample CT-OS (b) 
and Ethnicity Size {%) 



Non- Misc. 

Training JSA Non-Training 

OJT (c) Education (d) Only Services (e) 

(%) (%) (%) (%) 



All Employment 
Barrier Subgroups 

White, non-Hispanic 
Black, non-Hispanic 
Hispanic 

No Employment Barriers 

White, non-Hispanic 
Black, non-Hispanic 
Hispanic 

1 Employment Barrier 

Worked fewer than 13 weeks 
in 12 months before assignment 

White, non-Hispanic 

Black, non-Hispanic 

Hispanic 

No high school diploma or GED 
at assignment 

White, non-Hispanic 

Black, non-Hispanic 

Hispanic 

Receiving AFDC or other cash 
assistance at assignment 

White, non-Hispanic 

Black, non-Hispanic 

Hispanic 

2 Employment Barriers 

Worked fewer than 13 weeks 
in 12 months before assignment 
and no high school diploma or GED 
at assignment 

White, non-Hispanic 
Black, non-Hispanic 
Hispanic 



2,941 
1,432 
589 



1,162 
451 
158 



549 
282 
119 



336 
98 
93 



206 
152 
30 



195 
62 
65 



30.6 25.9 
50.5 *** 15.6 

38.7 *** 11.0 



*** 



9.1 
2.7 
15.8 



*** 



20.0 
19.7 
16.3 



29.9 28.1 2.2 23.5 

45.9 *** 18.6 *** 1.1 25.3 
43.0 *** 13.3 *** 10.8 *** 20.3 



37.2 
41.5 
41.2 



20.2 
45.9 
19.4 



39.3 
55.9 
46.7 



*** 



22.6 
18.1 
11.8 



35.4 
18.4 
12.9 



21.8 
15.8 
10.0 



24.1 
24.2 
20.0 



*** 



*** 
*** 



4.6 21.5 
2.8 22.7 
10.1 ** 22.7 



7.1 23.8 
5.! 18.4 
31.2 *** 18.3 



7.8 
1.3 
16.7 



17.5 
♦* 17.1 
10.0 



12.3 



15.9 



14.5 

11.5 *** 
18.2 *• 



16.4 
9.1 
12.7 



*** 



14.2 
14.9 
14.3 



13.4 
12.2 
18.3 



13.6 
9.9 
16.7 



15.9 



31.8 

19.4 * 1.6 ** 33.9 *** 21.0 
13.8 •** 26.2 *** 16.9 23.1 



(continued) 
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TABLE 5.3 (continued) 



Employment Barrier Subgroup (a) Sample CT-OS (b) 
and Ethnicity Size (%) 

Worked fewer than 13 weeks 
in 12 months before assignment 
and receiving AFDC or other cash 
assistance at assignment 



Non- 
Training JSA 
OJT (c) Education (d) Only 
(%) (%) (*) 



Misc. 
Non-Training 
Services (e) 

(*) 



White, non-Hispanic 


Oil 


AA 0 




1ft 0 
10. Z 






in a 




Black, non-Hispanic 


247 


68.4 


*** 


8.9 *** 


2.8 


*** 


10.5 


9.3 


Hispanic 


68 


61.8 


** 


5.9 ** 


4.4 


** 


4.4 


23.5 ** 


No high school diploma or GED 


















at assignment and receiving 


















AFDC or other cash assitance 


















at assignment 


















White, non-Hispanic 


118 


13.6 




19.5 


45.8 




13.6 


7.6 


Black, non-Hispanic 


43 


60.5 


*** 


9.3 


9.3 


*** 


11.6 


9.3 


Hispanic 


14 


35.7 


** 


7.1 


21.4 


* 


7.1 


28.6 ** 


All 3 Employment Barriers 


















White, non-Hispanic 


129 


22.5 




11.6 


48.8 




6.2 


10.9 


Black, non-Hispanic 


83 


63.9 


*** 


9.6 


8.4 


*** 


6.0 


12.0 


Hispanic 


37 


48.6 


*** 


2.7 


13.5 


*** 


5.4 


29.7 *** 



SOURCE: MDRC calculations from Background Information Form responses and from program enrollment 
and participation data from the 16 SDAs. 

NOTES: Calculations for this table are based on data for all white, black, and Hispanic adult enrollees in the 
18-month study sample. Enrollees from other ethnic groups were not included because of small sample sizes. 

The program service categories in this table are mutually exclusive; see notes b, c, d, and e below. 

For each employment barrier subgroup, within each program service category, a cbi-square test 
was applied to the difference in service receipt rates between white enrollees and black enrollees and 
between white enrollees and Hispanic enrollees. Statistical significance levels are indicated as 
* - 10 percent; ** «= 5 percent; *** *= 1 percent. 

(a) The barriers to employment include: not having a high school diploma or GED at the time of 
random assignment; having worked fewer than 13 weeks during the 12 months prior to random assignment; 
and receiving AFDC or other cash assistance at the time of random assignment. 

(b) The CT-OS service category includes enrollees who received CT-OS only, or CT-OS in 
combination with OJT, basic education, job search assistance, or miscellaneous program services. 

(c) The OJT service category includes enrollees who received OJT only, or OJT in combination 
with basic education, job search assistance, or miscellaneous program services. 

(d) The non-training education service category includes enrollees who received basic education 
only, or basic education in combination with job search assistance or miscellaneous program services. 

(e) The miscellaneous non-training services category includes enrollees who received miscellane- 
ous program services only, or miscellaneous program services in combination with job search assistance. 
Miscellaneous program services include such activities as assessment, job-readiness preparation, vocational 
exploration, job shadowing, and tiyout employment. In this table, miscellaneous program services also 
include work experience, which is identified as a separate program service in other reports on the National 
JTPA Study. 
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of the white adults and 46 percent of the black adults received CT-OS. The table also indicates 
that this difference was statistically significant at the 1 percent level 

Table 53 indicates that, with the exception of enroUees with limited work experience as 
their only employment barrier and enroUees with limited work experience who were also high 
school dropouts, black adults were more likely to receive CT-OS than white adults with the same 
employment barriers, Black and white adults in these two employment barrier subgroups were 
about equally likely to receive CT-OS. Also, within employment barrier subgroups that comprised 
over half of the black adults — those with no employment barriers and those whose only 
employment barrier included having limited work experience or no high school diploma or GED 
- black and white adults were about equally likely to receive basic education without training. 
Among all other employment barrier subgroups (particularly those that included public assistance 
receipt), black enroUees were much less likely to receive basic education without training. 

Table 53 also shows that there were no statisticaUy significant differences between black 
and white adults who received OJT in four of the eight employment barrier subgroups; in the 
other four subgroups, whites were more likely than blacks to receive OJT. 

Ethnic differences in service receipt may also be due to factors other than employment 
barriers. For example, 62 percent of the black adults and 80 percent of the Hispanic adults lived 
in 5 of the SDAs that were large and predominantly urban. These SDAs had access to large 
numbers of CT-OS providers, which may account for part of the ethnic difference in CT-OS 
receipt rates. Conversely, very few of the black and Hispanic adults lived in predominantly rural 
SDAs, some of which tended to focus on OJT services. This may account, in part, for the higher 
rates of OJT receipt among white adults. 14 

In order to investigate these and other hypotheses related to the association between service 
receipt rates and other individual characteristics, the analysis was extended to construct 
multivariate regression models that estimate service receipt rates while adjusting for differences 
that might be due to a variety of individual background and SDA characteristics. The following 
results are based on a regression model that adjusts for differences among ethnic groups that were 
associated with age, gender, AFDC history, household composition, random-assignment date, Food 



14 It is important to note that an analysis of variation among the 16 study sites did not reveal a 
consistent relationship between service emphasis and urban/rural distinctions. In fact, even SDAs that 
encompassed mostly rural areas and emphasized OJTs had at least one moderate-sized city that accounted 
for a significant portion of the study sample. 
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Stamp receipt at random assignment, previous enrollment in job club, having a high school 
diploma rather than a GED, and SDA, as well as the employment barrier combinations listed in 
Table 53. The results from this model indicate that the ethnic differences in service receipt rates 
decreased markedly when estimated with this model For example, the regression-adjusted 
percentage of white adults who received CT-OS was 35 percent - compared with 31 percent of 
the black adults - and the regression-adjusted percentage of white adults who received OJT was 
25 percent - compared with 22 percent of the black adults. For this particular regression model 
specification, the white/black difference in CT-OS receipt was not judged to be statistically 
significant and the white/black difference in OJT receipt was judged to be statistically significant 
only at the 10 percent level. The results from this model indicate that ethnic differences in 
service receipt can be explained on the basis of site differences in service emphasis and 
differences in service receipt among subgroups defined by characteristics other than ethnicity. 
However, other model specifications yielded differences that were statistically significant. 

Readers should exercise caution in interpreting the multivariate regression results because 
they are sensitive to ihe regression model specification used. Also, this type of statistical 
modeling is very difficult to translate into policy-relevant and practical conclusions. SDAs must 
serve real people rather than individuals (reflected in the regression model) who are the same on 
all characteristics except ethnicity. Therefore, the analysis shown in Table 53 is probably the 
most realistic picture of ethnic differences in service receipt in the study sites. 

The analysis discussed in this section provides a further context for interpreting the ethnic 
differences in service receipt rates displayed in Table 5.1. In short, the discussion highlighted the 
fact that white adults possessed different employment barriers than black or Hispanic adults and 
that these factors do account for some of the differences in service receipt (particularly OJT 
receipt) but not all of the differences. The analysis also showed that other measurable differences 
among ethnic groups (such as the type of SDA they lived in or their family status), help explain 
more of the service receipt differences depending on various aspects of the multivariate regression 
modeling technique used; but again, differences may remain. 

Several other factors not analyzed here are likely to play a role in the distribution of JTP A 
services among ethnic (and other) subgroups of enrollees. First, the data collected for the 
National JTPA Study, along with the analysis discussed in this report, did not include other 
individual characteristics (such as basic skills level, motivation, and access to other opportunities 
in the labor market) that may be associated with ethnicity and that may account for differences 
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in service receipt rates. Second, the analysis was also not able to account for the particular 
service preferences of individual JTPA applicants. Third, the analysis Hissed in tb ; s report 
cannot discount the possibility that employers discriminate on the basis of ethnicity in hiring or 
SDA staff discriminate in service allocation decisions. 

Finally, the analysis discussed in this chapter focuses only on individuals who proceeded 
through the three stages - discussed in Chapter 3 - in the JTPA service delivery process at 
which JTPA applicants are screened for service receipt At each stage, it is likely that a 
combination of individual characteristics, individual preferences, and SDA and service provider 
practices (which, in some cases, may amount to racial discrimination) account for differences m 
the way JTPA services are distributed among ethnic groups. 

n. The Length of Enrollment for Adults 

One potentially important indicator of the likely influence of the JTPA treatment is the 
length of that treatment. This analysis uses the number of months that adults were enrolled in 
JTPA during the 18-month follow-up period as the primary measure of service intensity. The 
assumption here is that those who participated for substantial periods of time were more likely 
to accumulate employable skills than those who participated for shorter periods of time. 15 

A. The Relationship Between Length of Enrollment and Services Received 

The top row in Table 5.4 lists the median length of enrollment for all adult enrollees and 
for adult enrollees in each of the five service categories. 16 It shows that, overall, the median 



IdeaUy, the analysis would also include other measures of service quality and intensity such as hours 
per week, the skills level of classroom training or OJT, the orientation to the skills levels of the 
participants, coordination of occupational training and basic skills remediation, and level of job-relevant 
information in the curricula. The National JTPA Study did not provide the level of resources required to 
undertake an investigation of these measures in each of the SDAs in the study. For a discussion of these 
issues, see U.S. Department of Labor, 1991. 

16 Length of enrollment for individuals in a particular service category is not necessarily equivalent to 
the number of months they spent in a specific service. Length of enrollment is defined here as the total 
time during the follow-up period that an individual was counted by SDA staff as officially enrolled. In a 
few cases, this included a short period in the middle of a sequence of services in which SDA staff placed 
an individual in a "hold" status. 

The median is the most common measure of central tendency used in analyses of data on length 
of stay or duration. Here, median enrollment duration is defined as 'the number of months of enrollment 
that elapsed before half of the adult enrollees completed their JTPA participation. For those whose 
enrollment records contained a valid start and end date, length of enrollment was simply the total number 
of days between the start and end dates, which was divided by 30 to obtain the number of months enrolled. 

(continued...) 
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length of enrollment for adult enrollees was just over three months* The top row in Table 5.4 
also indicates dramatic differences in length of enrollment among adults who received particular 
categories of services. For example, the median length of enrollment among adults who received 
CT-OS was almost twice that of adults who received OJT and almost five times that of adults who 
received job search assistance only. 17 

The content, scheduling, and specifications of particular services usually determine their 
length. For example, the duration of most classroom training services usually corresponds to an 
academic calendar that divides the year into trimesters. A review of typical contracts between the 
SDAs in the study and the vendors who provided services indicates that most classroom training 
placements were specified for four months or more. A review of typical OJT contracts indicates 
that training services were to last three months or fewer depending on the SDA and, in some 
cases, the skills level of the position. 

B. Subgroup Differences in Length of Enrollment 

The remainder of Table 5.4 lists selected subgroups of adult enrollees and the median 
length of enrollment (in months) for each subgroup, both overall and by the category of services 
they received. Statistical tests were performed to determine whether there were significant 
differences in median length of enrollment between the first subgroup listed under each 
characteristic and each of the other subgroups. For example, among the subgroups def ned by 
ethnicity, the table indicates that there was a statistically significant difference between the median 



"(...continued) 

While all of the enrollees had valid start dates, approximately 10 percent of them had no end dates. 
The data files from the SDAs did not identify whether these constituted coding errors or ongoing 
enrollments. In general, however, all of the exits from JIPA that were coded on the data files took place 
within 12 months of enrollment. In other words, there were no exits from JTPA after 12 months even 
though, depending on the subgroup and the primary service received, more than 10 percent of the sample 
appeared to participate for 12 months or longer. This led to the conclusion that many of the enrollments 
that lasted longer than 12 months without a formal termination represented coding errors. In order to 
include these sample members in the analyses without ending their enrollment periods arbitrarily, these 
enroUment durations were coded as "open" after 12 months. This means that after month 12 they were no 
longer included in the analysis of the likelihood of leaving JTPA in a particular month given that their 
enrollment periods covered the previous 12 months. (See Appendix C for more information about the data.) 

These differences in length of enrollment among adults in each of the service categories account for 
similar differences in length of enrollment among adults in each of the recommended service strategies. 
For example, adult enrollees recommended for the classroom training service strategy had the longest 
median length of enrollment at 4.6 months, and those recommended for the OJT/JSA service strategy had 
the shortest median length of enrollment at 2,2 months. Adult enrollees recommended for the other 
services strategy had a median length of enrollment of 2.4 months. 
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length of enrollment for all white adults (3.1 months) and the median length of enrollment for 
all Hispanic adults (4*0 months)* The table also indicates that there was no statistically significant 
difference between the median length of enrollment for white adults and the median length of 
enrollment for black adults (3.0 months)* 18 

Many of the differences in median length of enrollment among subgroups are associated 
with the differences in service receipt rates exhibited in Table 5*1. Subgroups that were likely to 
enroll in classroom-based services such as CT-OS and basic education without training tended to 
have the longest enrollments while those likely to enroll in employment-based services such as 
OJT and job search assistance only had the shortest enrollments. Even among enrollees who 
received services in the same category, some subgroups had longer enrollments than others. In 
many cases, however, these differences were due to the fact that those with longer enrollments 
were more likely to receive two or more services than those with shorter enrollments. 

The findings presented in Table 5*4 help illustrate the association between subgroup 
differences in length of enrollment and the types and number of services enrollees received. For 
example, they show that women remained in JTPA longer than men. The median length of 
enrollment for men was 25 months compared with a median length of enrollment of 3.6 months 
for women. This difference is due, in part, to differences in the types and numbers of services 
they received. Recall from Table 5*1 that women tended to receive CT-OS and men tended to 
receive OJT and job search assistance only. However, Table 5*4 also shows that the median 
length of enrollment for women who received CT-OS was one month longer than the median 
length of enrollment for men who received CT-OS. This is primarily because women were more 
likely than men to combine CT-OS with other services (particularly job search assistance and 
miscellaneous services). 

Table 5*4 indicates several other subgroup differences in length of enrollment that may not 
be associated only with differences in the types or number of services received. First, the median 
length of enrollment for black adults who received CT-OS or OJT was shorter than that for white 
adults who received those services. This suggests that black adults may have been more likely to 
leave than white adults — either for positive reasons such as finding a job or for other reasons 
such as lack of interest. It may also be that the training that black adults received was oriented 



18 Unless otherwise noted, discussion of median length of enrollment differences among subgroups 
focuses on those that are statistically significant at the 10 percent level or lower. 
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toward lower-skill occupations and therefore, by design, shorter than that received by white adults. 
Interestingly, Hispanic adults who received CI-OS or OJT had longer enrollments than white 
adults who received these services. 

Among adults who received CT-OS or OJT, high school dropouts had shorter enrollments 
than those who had a high school diploma or GED at the time of their JTPA application. This 
may signify that those who left school previously may have continued this pattern by not 
completing their training or that the training was, by design, shorter than that received by 
graduates and GED recipients. However, the median length of enrollment for high school 
dropouts who received basic education without training was actually two months longer than that 
for graduates who received basic education. Finally, Table 5.4 shows that the median length of 
enrollment for public assistance recipients in all service categories was longer than that for those 
who were not receiving public assistance at the time of their JTPA application. In the absence 
of supplementary support from JTPA it is likely that cash public assistance payments provided 
the financial support necessary to sustain (these adults in their JTPA services for longer periods 
of time. 
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CHAPTER 6 

SERVICES RECEIVED BY 
OUT-OF-SCHOOL YOUTH ENROLLEES 



This chapter describes the distribution of JTPA services among out-of-school youths in the 
18-month treatment group sample who were officially enrolled in JTPA at some point during the 
18-month follow-up period 1 like Chapter 5, it is organized around the types of JTPA services 
that out-of-school youths actually received and the length of their enrollments in the program 
rather than around the services for which they were recommended and their status within the 
research design (Le., a!! treatment group members vs. those who were actually enrolled). For the 
analysis in this chapter, youth enrollees have been grouped according to the five mutually exclusive 
service categories described in Chapter 5. The analysis focuses on service receipt and length of 
enrollment for the out-of-school youth target groups (male and female) and for other key 
subgroups of youth enrollees. Like the information in Chapter 5, the information in this chapter 
provides both a context for understanding program impacts and a perspective on how JTPA 
resources were distributed among some of the key demographic groups that the SDAs in the study 
tried to target for services. 

Following is an overview of key findings in the chapter: 

• Just over half of the out-of-school youths received some form of occupational 
training - either in the classroom or on-the-job — but a substantial 
percentage received basic education without training. Thirty-six percent of the 
out-of-school youth enrollees received classroom training in occupational skills 
(CT-OS) and an additional 16 percent received on-the-job training (OJT). 
Thirty-seven percent of the out-of-school youths received basic education or 
miscellaneous services without training. The remaining 11 percent received job 
search assistance only. 

• Youths with two or three of the employment barriers analyzed (high school 
dropout, limited work experience, and cash public assistance recipient) were 
more likely to receive CT-OS or basic education without training and less 
likely to receive OJT or Job search assistance only than those with no 
employment barriers. Over 60 percent of the youths with two or three 



l As noted in earlier chapters, treatment group members were identified as "officially enrolled in JTPA" 
if the local SDA staff established an enrollment record on their JTPA management information system 
(MIS) during the 18-month follow-up period. 
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employment barriers received either CT-OS or basic education without training 
compared with 43 percent of thcxe with no employment barriers. Forty-three 
percent of the youths with no employment barriers received OJT or job search 
assistance only compared with less than 15 percent of those with two or three 
barriers. 

• Female youths were more likely to receive CT-OS and less likely to receive 
OJT or job search assistance only thai male youths. Employment barriers 
accounted for some, but not all, of the differences in service receipt between 
male and female youths. Forty-three percent of the female youths received CT- 
OS compared with 28 percent of the male youths. Thirty-one percent of the 
male youths received OJT or job search assistance only compared with 22 
percent of the female youths. In general, employment barriers accounted for 
much of the male/female difference in OJT and job search assistance only 
receipt rates, but only part of the difference in CT-OS receipt rates. 

• White youths were more likely than black youths to receive OJT or Job search 
assistance only and less likely to receive CT-OS or basic education. 
Employment barriers accounted for some, but not all, of the differences in 
service receipt between white and black youths* Thirty-five percent of the white 
youths received OJT or job search assistance only compared with 16 percent of 
the black youths. By contrast, 65 percent of the black youths received either 
CT-OS or basic education without training compared with 46 percent of the 
white youths. 

• The median length of enrollment for youths in the Ave service categories 
ranged from over five months to less than one month. Overall, the median 
length of enrollment for out-of-school youths was 3.6 months. The median 
length of enrollment was 53 months for those who received CT-OS, 3.1 months 
for those who received OJT, and .8 months for those who received job search 
assistance only. 

The remainder of the chapter expands on these findings. 



I. Service Receipt in the Five Service Categories 

After describing the overall distribution of services among youths, this section presents three 
levels of analysis of the types of services received by out-of-school youth enrollees. It examines 
the services received by each of the two youth target groups (male youths and female youths), 
analyzes differences in the distribution of the five key categories of services among selected 
subgroups of youths, and, finally, examines whether service receipt differences among youths with 
various combinations of employment barriers account for gender and ethnic differences in service 
receipt 
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The top row in Table 6.1 shows that 36 percent of all youths received CT-OS and an 
additional 16 percent received OJT. Overall, therefore, 52 percent of the out-of-school youths 
received some type of occupational training - either in the classroom or on-the-job. The table 
also shows that 37 percent of the out-of-school youths received basic education or miscellaneous 
services without occupational training. The remaining 11 percent of the out-of-school youths 
received job search assistance only. 

A comparison of the service receipt rates listed in the top row of Tables S.l and 6.1 
highlights several important differences between adults and youths. 2 First, youths were somewhat 
less likely than adults to receive some form of occupational training that included either CT-OS 
or OJT. Fifty-two percent of the youths received either CT-OS or OJT compared with 59 
percent of the adults. Second, youths were more likely than adults to receive basic education and 
miscellaneous services without occupational training. Table 6.1 shows that 37 percent of the 
youths received these non-training services compared with only 22 percent of the adults (Table 
5.1). In particular, 19 percent of the youths received basic education without training compared 
with 8 percent of the adults. Third, youths were less likely than adults to receive job search 
assistance only. Table 6.1 shows that 11 percent of the youths received job search assistance as 
their only service, compared with 19 percent of the adults (Table 5.1). Finally, although not 
displayed in the tables, youths were more likely than adults to receive two or more services; 35 
percent of the youths received two or more services compared with only 26 percent of the adults. 

The remainder of Table 6.1 lists selected subgroups of youth enrollees and the percentage 
of each subgroup who received each category of services. Statistical tests were performed to 
determine whether there were significant differences in service receipt rates between the first 
subgroup listed under each characteristic and each of the other subgroups. For example, among 
the subgroups defined by ethnicity, the table indicates that there was a statistically significant 
difference between the percentage of white youths who received CT-OS (30 percent) and the 
percentage of Hispanic youths who received CT-OS (61 percent). The table indicates that there 
was no statistically significant difference between the percentage of white youths who received 
CT-OS and the percentage of black youths who received CT-OS (33 percent). 3 



2 Note that service receipt differences between adults and youths were not tested for statistical 
significance. As a result, the discussion of adult/youth differences in service receipt is confined to the 
largest differences between the percentages in Table 5.1 and in Table 6.1. 

3 Unless otherwise noted, discussion of service receipt differences among subgroups focuses on those 
that are statistically significant at the 10 percent level or lower. 
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TABLE 6.1 



SERVICE RECEIPT RATES FOR OUT-OF-SCHOOL YOUTH ENROLLEES, 
BY SELECTED BASELINE CHARACTERISTICS 



Characteristic 
and Subgroup 



Non- Misc. 

Training JSA Non-Training 

Sample CT-OS (a) OJT (b) Education (c) Only Services (d) 

Size (%) (*) (*) (*) (*) 



All Out-of-School Youth Enrollees 2, 147 



36.4 



15.5 



18.6 



10.9 



18.6 



Target Group 
Male Youths 
Female Youths 



959 
1,188 



28.1 
43.1 



18.0 
13.4 



*** 



17.9 
19.2 



13.2 
9.0 



*** 



22.7 
15.3 



*** 



Age 

16-19 
20-21 



1,322 
825 



35.7 
37.5 



12.5 22.1 10.7 19.1 

20.2 *** 13.1 *** 11.3 17.9 



Ethnicity 
White, non-Hispanic 
Black, non-Hispanic 
Hispanic 
Other 



1,138 
588 
375 
46 



Education 

High school diploma or GED 885 
No high school diploma or GED 1 , 173 



Public Assistance Receipt 
Not receiving AFDC or other cash 

assistance at assignment 
Receiving AFDC or other cash 

assistance at assignment 

Barriers to Employment (e) 
None 
1 
2 
3 



1,479 



458 



499 
798 
638 
207 



29.9 21.4 16.0 13.1 19.7 

33.0 7.0 *** 31.8 *** 9.4 ** 18.9 

60.8 *** 10.9 *** 6.9 *** 7.2 *** 14.1 

41.3 15.2 10.9 6.5 26.1 



44.1 24.0 2.6 13.2 16.2 

30.6 *** 8.8 *♦* 30.8 *** 8.8 *** 21.1 *** 



Work Experience 
Worked 13 weeks or more 

in 12 months before assignment 877 35.6 
Worked fewer than 13 weeks 

in 12 months before assignment 1,064 36.9 



35.9 
37.8 



20.9 11.1 14.1 18.4 

11.2 *** 25.1 *** 7.9 *** 18.9 

17.6 16.0 12.6 17.8 

10.0 *** 24.9 *** 7.6 *** 19.7 



27.1 3.0 15.4 14.8 

16.4 *** 13.5 *** 12.9 19.3 ** 

33.5 ** 9.1 *** 30.4 *** 6.7 *** 20.2 ** 

31.4 ** 3.4 *** 39.6 *** 5.3 *** 20.3 * 



39.7 
37.8 



(continued) 
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TABLE 6.1 (continued) 



Not- Misc. 
Training JSA Non-Training 



Characteristic 


Sample 




OJT (b) Education (c) 






and Subgroup 


Size 


(*) 






{%) 


(%) 


AFDC History 














Never received AFDC 


1,783 


35.3 


15.7 


17.8 


11.9 


19.3 


Received AFDC less than 














2 years (f) 


Zo7 


JO.O 


15.0 


24.0 ** 


U.H 


*** 16.1 


Received AFDC 2 years 














or more (f) 


82 


52.4 


11.0 


20.7 




* 11 n 


Household Composition 












16.5 


Spouse present 


248 


32.7 


20.6 


19.0 


11.3 


No spouse present, child present 


437 


43.0 *** 


14.2 ** 


20.8 


8.7 


13.3 


No spouse present, no child 














present 


1,316 


35.3 


15.6 * 


16.3 


11.8 


21.1 



SOURCE: MDRC calculations from Background Information Form responses and from program enrollment 
and participation data from the 16 SDAs. 

NOTES: Calculations for this table are based on data for all out-of-school youth enrollees in the 18-month study 
sample. The total sample size may vary among characteristics because of missing data for some enrollees. 

The program service categories in this table are mutually exclusive; see notes a, b, c, and d below. 

For each characteristic, within each program service category, a chi-square test was applied to the 
difference in service receipt rates between the first subgroup listed and, separately, each of the remaining subgroups. 
Statistical significance levels are indicated &s * ~ 10 percent; ** = 5 percent; *** = 1 percent. 

(a) The CT-OS service category includes enrollees who received CT-OS only, or CT-OS in 
combination with OJT, basic education, job search assistance, or miscellaneous program services. 

(b) The OJT service category includes enrollees who received OJT only, or OJT in combination 
with basic education, job search assistance, or miscellaneous program services. 

(c) The non-training education service category includes enrollees who received basic education 
only, or basic education in combination with job search assistance or miscellaneous program services. 

(d) The miscellaneous non-training services category includes enrollees who received miscellane- 
ous program services only, or miscellaneous program services in combination with job search assistance. 
Miscellaneous program services include such activities as assessment, job-readiness preparation, vocational 
exploration, job shadowing, and tryout employment. In this table, miscellaneous program services also 
include work experience, which is identified as a separate program service in other reports on the National 
JTPA Study. 

(e) The barriers to employment include: not having a high school diploma or GED at the time of 
random assignment; having worked fewer than 13 weeks during the 12 months prior to random assignment; 
and receiving AFDC or other cash assistance at the time of random assignment. 

(f) The AFDC receipt may not have been continuous. 
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A, Service Receipt Rates for Each of the Youth Target Groups 

Table 6.1 shows that female youths were more likely than male youths to receive CT-OS, 
but they were less likely to receive employment-based services such as OJT and job search 
assistance only. Forty-three percent of the female youths received CT-OS compared with 28 
percent of the male youths, and 18 percent of the male youths received OJT compared with 13 
percent of the female youths. The large difference in the percentage of female and male youths 
who received CT-OS produced an overall difference in the percentage who received any form of 
occupational training. In total, 57 percent of the female youths received either CT-OS or OJT, 
while 46 percent of the male youths did so. However, 31 percent of the male youths received 
some type of employment-based service in the form of OJT or job search assistance only. This 
was calculated by adding the percentage of male youths who received OJT (18 percent) to the 
percentage receiving job search assistance as their only service (13 percent). By contrast, 22 
percent of the female youths received either OJT or job search assistance only. Further analysis 
of the target group differences in service receipt rates is discussed later in the chapter. 

B. Service Receipt Rates for Subgroups of Youth Enrollees 

This section discusses the service receipt rates for subgroups of out-of-school youth 
enrollees. Although there were large differences in service receipt rates for male and female 
youths, many of the other subgroup differences in service receipt are similar for these two target 
groups. Subgroup findings that differ for male and female youths (e.g., those related to 
employment barriers) are indicated in the text, and Appendix B provides separate subgroup tables 
for male and female youths. 

1. Age subgroups. Youths aged 16-19 were just as likely to receive CT-OS, job search 
assistance only, and miscellaneous non-training services as youths aged 20-21. However, 22 
percent of the out-of-school youths aged 16-19 (compared with only 13 percent of those aged 20- 
21) received basic education without training. These are the youths least likely to have returned 
to school before their JTPA enrollment to obtain their high school diploma or GED. JTPA 
afforded them the opportunity to do that. Finally, the older youths were more likely to receive 
OJT than the younger youths. Here, the older youths may have accumulated more work 
experience and thus become more attractive to employers trying to Gil OJT slots. 

2. Ethnic subgroups . Table 6.1 shows that 33 percent of the black youths and 30 
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percent of the white youths received CT-OS and that this difference was not statistically 
significant It also shows that white youths were three times more likely to receive OJT than 
black youths. 4 Fifty-one percent of the white youths received occupational training in the form 
of CT-OS or OJT, compared with 40 percent of the black youths. This contrasts with the 
findings for adults, which show that black men and women were more likely to receive 
occupational training than white men and women. Also, unlike the adults, black youths were 
twice as likely as white youths to receive basic education without training, which included basic 
education and GED preparation. This is surprising given that similar percentages of black and 
white youths (56 and 51 percent, respectively) had no high school diploma or GED at the time 
of JTPA application. 

Hispanic youths were twice as likely as white youths to receive CT-OS, but were about half 
as likely to receive the employment-based services including OJT and job search assistance only. 
Although Table 6.1 shows that only 7 percent of the Hispanic youths received basic education 
without training, an additional 28 percent 5 received basic education in combination with either 
CT-OS or OJT. Recall that basic education services include classes in English as a second 
language (ESL). This finding suggests that, like the Hispanic adults, Hispanic youths may have 
used JTPA support as a means of improving their English. Unlike the adults, however, Hispanic 
youths were more likely to combine basic education with occupational training to become more 
competitive in the labor market 

Later in the chapter, the discussion returns to an analysis of the extent to which differences 
in service receipt rates associated with employment barriers account for differences in service 
receipt rates among ethnic groups. 

3. Em ployment barrier subgroups. This section focuses on the receipt of JTPA services 
for subgroups of youth enrollces defined by three employment barriers: those with no high school 
diploma or GED at the time of random assignment (referred to as high school dropout); 6 those 
who worked fewer than 13 weeks in the year prior to random assignment (referred to as limited 
work experience); and those who were receiving AFDC or other cash assistance at the time of 



^is difference is even larger among male youths, where 25 percent of the white youths received OJT 
compared with only 6 percent of the black youths (see Appendix B). 

5 In Table 6.1, these individuals are included among the 61 percent of Hispanic youths who received 
CT-OS and the 11 percent who received OJT. 

*Note that although the sample includes only out-of-school youths, more than half of the youths 
obtained a high school diploma or GED before they applied for JTPA services (see Table 2.12). 
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random assignment (referred to as cash public assistance recipient). 

The general association between employment barriers and service receipt can be seen in the 
panel of Table 6.1 that lists subgroups of youth enrollees defined by the number of employment 
barriers they possessed. Compared with youths with none of the employment barriers, those with 
two or three barriers were much more likely to receive basic education without training and much 
less likely to receive OJT or job search assistance only. The following analysis of the individual 
employment barrier subgroups helps illuminate some of the underlying relationships between 
service receipt and the needs of out-of-school youths. For example, the effect of not having a 
high school diploma or GED accounts for much of the difference in receipt of CT-OS between 
those with multiple employment barriers and those with no employment barriers. 

Table 6.1 indicates that not having an educational credential was a key barrier to receiving 
occupational training, including CT-OS or OJT. Forty-four percent of the high school graduates 
and GED recipients received CT-OS compared with 31 percent of the high school dropouts. 
Also, Table 6.1 shows an especially large difference in OJT receipt rates between those with a 
high school diploma or GED and high school dropouts; 24 percent of the graduates and GED 
recipients received OJT compared with only 9 percent of the dropouts. These findings suggest 
that classroom training providers and employers were likely to be cautious about investing in 
youths who had not yet demonstrated educational persistence by obtaining a high school diploma 
or GED. 

At the same time, however, high school dropouts were more than ten times as likely as 
graduates and GED recipients to receive basic education services without training. Given that 
these youths did not receive subsequent JTPA-funded CT-OS or OJT services during the follow- 
up period, the payoff on their investment in basic education or GED preparation, at least in 
terms of further training, is likely to occur later than it is for those who received CT-OS or OJT 
during the follow-up period. 

Not surprisingly, youths with limited work experience were less likely to receive employment- 
based services such as OJT or job search assistance only than those who worked 13 weeks or 
more during the year prior to JTPA application. However, youths with limited work experience 
were more likely to receive CT-OS (although this was not statistically significant) and basic 
education without training. 

Fully 87 percent of the youths who were receiving cash public assistance were either hi£h 
school dropouts or had limited work experience, or both, which may have required them to obtain 
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further education before entering the labor market In fact, Table 6.1 shows that compared with 
those who were not receiving cash welfare payments, youths who were receiving AFDC or other 
cash assistance at the time of JTPA application were more likely to receive basic education 
without training and were less likely to receive employment-based services such as OJT and job 
search assistance only. 

4. AFDC history and household composition subgroups. Table 6.1 shows that long- 
term AFDC recipients and single parents - those most likely to be AFDC recipients - were 
more likely to receive CT-OS than those who had never received AFDC and those who were 
married, respectively. 7 

C Further Discussion of Kcv Findings 

Three sets of findings stand out from the analyses discussed in the previous sections: gender 
differences in service receipt, ethnic differences in service receipt, and the association between 
the three employment barriers analyzed and the types of services that out-of-school youths 
received. As is the case with adults, these issues have been the subject of other recent reports 
on JTPA. 8 This section addresses the question of whether differences in service receipt rates 
associated with the employment barriers account for differences in the services received among 
gender and ethnic groups. Specifically, the analysis examines gender and ethnic differences in 
service receipt within subgroups of enrollees defined by the various combinations of the three 
employment barriers discussed in the previous section (high school dropout, limited work 
experience, and cash public assistance recipient). This will permit a further investigation of the 
hypothesis that male and female youths (or black and white youths) with the same employment 
barriers are equally likely to receive CT-OS. 

Because many of the black and Hispanic youths were concentrated in the larger urban 
SDAs, the analysis of ethnic differences in service receipt is expanded further. Specifically, ethnic 
differences in service receipt were tested within a multivariate regression framework that adjusted 



7 Given the small number of male youths who had any history of AFDC receipt, the percentages for 
subgroups of this characteristic displayed in Table 6.1 reflect service receipt primarily for female youths. 

8 In particular, one study analyzed data on a random sample of about 5,000 youths from 63 SDAs and 
found that: (1) JTPA served less job-ready youths in roughly the same proportion as their representation 
in the eligible population (as estimated by the U.S. Census Bureau's Current Population Survey); (2) more 
job-ready youths tended to receive occupational training (either in the classroom or on the job), while the 
less job-ready were more likely to be enrolled in non-occupational training; (3) black youths were more 
likely to receive either non-occupationil training or job search assistance only, and less likely to be enrolled 
in occupational training. See U.S. General Accounting Office, 1990. 

Ill 
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for other differences in service receipt that may have been due to the distribution of ethnic 
groups among different types of SDAs or to individual characteristics such as age, household 
composition, AFDC history, and previous earnings that may also be associated with ethnicity. 

L Further analysis of gender differences in service receipt The top panel of Table 
62 is taken directly from Table 6.1 and shows that, overall, male youths were more likely to 
receive OJT or job search assistance only and less likely to receive CT-OS than female youths. 
In part, differences in the types of services male and female youths received can be traced to 
differences in the services for which they were recommended at assessment (see Table 4.1). For 
example, 44 percent of the female youths and 30 percent of the male youths were recommended 
for CT-OS. Also, 33 percent of the male youths were recommended for OJT and an additional 
5 percent were recommended for job search assistance only. By contrast, 23 percent of the 
female youths were recommended for OJT and an additional 2 percent were recommended for 
job search assistance only. 

Like the differences between adult men and women highlighted in Chapter 5, service receipt 
(as well as service recommendation) differences between male and female youths may also be 
explained by differences in the types of employment barriers they possessed For example, 42 
percent of the female youths had two or three of the employment barriers analyzed compared 
with 32 percent of the male youths (see Table 2.13). Also, 61 percent of the female youths had 
limited work experience — an attribute, as discussed above, associated with lower receipt rates of 
OJT or job search assistance only - compared with only 47 percent of the male youths. 

In general, employment barriers accounted largely for target group differences in OJT and 
job search assistance only receipt, but only partly for target group differences in CT-OS receipt 
The emaining panels in Table 62 display service receipt rates for male and female youths within 
subgroups defined by each combination of the three employment barriers. The statistical 
significance levels indicate whether the percentage of male youths who received a particular 
category of services differed from the percentage of female youths who received that category of 
services. For example, the second panel indicates that among youths with no employment 
barriers, 47 percent of the female youths received CT-OS compared with 31 percent of the male 
youths. The statistical significance level indicates that this difference was statistically significant 
at the 1 percent level. Because so few male youths were receiving cash public assistance at the 
time of their JTPA application and because cash public assistance receipt was highly correlated 
with the other two employment barriers, the following discussion focuses on employment barrier 
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TABLE 6.2 



SERVICE RECEIPT RATES FOR OUT-OF-SCHOOL YOUTH ENROLLEES, 
BY EMPLOYMENT BARRIER SUBGROUP AND TARGET GROUP 



Employment Barrier Subgroup (a) Sample CT-OS (b) 
and Target Group Size (%) 



Non- 
Training JSA 
On (c) Education (d) Only 
(%) (%) (%) 



Misc. 
Non-Training 
Services (e) 
(*) 



All Employment 
Barrier Subgroups 

Male youths 
Female youths 

No Employment Barriers 

Male youths 
Female youths 



959 
1,188 



227 
272 



28.1 
43.1 



30.8 
47.1 



*** 



*** 



18.0 17.9 13.2 22.7 
13.4 *** 19.2 9.0 *** 15.3 *** 



34.4 
21.0 



*** 



1.8 
4.0 



16.7 
14.3 



16.3 
13.6 



1 Employment Barrier 

Worked fewer than 13 weeks 
in 12 months before assignment 

Male youths 

Female youths 



132 
169 



32.6 
53.3 



22.7 
18.3 



5.3 
4.7 



18.2 
8.9 



21.2 
14.8 



No high school diploma or GED 
at assignment 

Male youths 

Female youths 

Receiving AFDC or other cash 
assistance at assignment 

Male youths 

Female youths 



254 
170 



17 
56 



27.2 
39.4 



*** 



17.6 

53.6 ** 



14.6 
10.6 



11.8 
23.2 



22.8 
19.4 



5.9 
1.8 



13.0 
12.9 



23.5 
8.9 



22.4 
17.6 



41.2 
12.5 
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2 Employment Barriers 

Worked fewer than 13 weeks 

in 12 months before assignment 

and no high school diploma or GED 

at assignment 
Male youths 240 
Female youths 220 

Worked fewer than 13 weeks 

in 12 months before assignment 

and receiving AFDC or other cash 

assistance at assignment 
Male youths 10 
Female youths 86 



27.1 
33.6 



30.0 
61.6 
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8.7 
5.9 



0.0 
19.8 



I PS 



31.7 
40.0 



(0 



7.5 
4.5 



25.0 
15.9 



0.0 30.0 
7.0 (f) 3.5 *** 



40.0 
8.1 *** 

(continued) 



TABLE 6.2 (continued) 



Non- Misc. 

Training JSA Non-Training 

Employment Barrier Subgroup (a) Sample CT-OS (b) OJT (c) Education (d) Only Services (e) 

and Target Group Size (%) (%) (%) (*) (%) 

No high school diploma or GED 
at assignment and receiving 
AFDC or other cash assistance 
at assignment 

Male youths 33 18.2 9.1 27.3 9.1 36.4 

Female youths 49 26.5 8.2 30.6 12.2 . 22.4 

All 3 Employment Barriers 

Male youths 45 22.2 2.2 37.8 8.9 28.9 

Female youths 162 34.0 3.7 40.1 4.3 17.9 



SOURCE: MDRC calculations from Background Information Form responses and from program 
enrollment and participation data from the 16 SDAs. 

NOTES: Calculations for this table are based on data for all out-of-school youth enrollees in the 18-month 
study sample. 

The program service categories in this table are mutually exclusive; see notes b v c, d, and e below. 

For each employment barrier subgroup, within each program service category, a chi-square test was 
applied to the difference in service receipt rates between male youths and female youths. Statistical significance 
levels are indicated as * ~ 10 percent; ** « 5 percent; *** « 1 percent. 

(a) The barriers to employment include: not having a high school diploma or GED at the time of 
random assignment; having worked fewer than 13 weeks during the 12 months prior to random assignment; 
and receiving AFDC or other cash assistance at the time of random assignment. 

(b) The CT-OS service category includes enrollees who received CT-OS only, or CT-OS in 
combination with OJT, basic education, job search assistance, or miscellaneous program services. 

(c) The OJT service category includes enrollees who received OJT only, or OJT in combination 
with basic education, job search assistance, or miscellaneous program services. 

(d) The non-training education service category includes enrollees who received basic education 
only, or basic education in combination with job search assistance or miscellaneous program services. 

(e) The miscellaneous non-training services category includes enrollees who received miscellane- 
ous program services only, or miscellaneous program services in combination with job search assistance. 
Miscellaneous program services include such activities as assessment, job-readiness preparation, vocational 
exploration, job shadowing, and try out employment. In this table, miscellaneous program services also 
include work experience, which is identified as a separate program service in other reports on the National 
JTPA Study. 

(0 A chi-square test was inappropriate because of low expected cell frequencies. 
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combinations that do not include cash public assistance receipt 

Table 62 shows that the target group difference in OJT receipt was statistically significant 
only among youths with no employment barriers. In other words, among youths with 
combinations of employment barriers that included being a high school dropout or having limited 
work experience, male and female youths were about equally likely to receive OJT. Similarly, 
although male youths were more likely than female youths to receive job search assistance only, 
the differences persisted only among youths with limited work experience. 9 However, recall that, 
in general, youths with these employment barriers were less likely to receive OJT or job search 
assistance only than those with no employment barriers. 

Among youths with no employment barriers and youths with only one employment barrier, 
female youths were more likely to receive CT-OS than male youths. As Table 62 indicates, 
among youths who were high school dropouts and had limited work experience, however, there 
were no statistically significant differences in CT-OS receipt between male and female youths. 

2. Further analysis of ethnic differences fa service receipt The top panel of Table 
63 is taken directly from Table 6.1 and displays the overall finding, discussed in the previous 
section, that white youths were more likely than black youths to receive OJT or job search 
assistance only and less likely to receive basic education without training. like the ethnic 
differences in service receipt among adults, service receipt differences among ethnic subgroups 
of out-of-school youths may be associated with employment barriers. Black youths were much 
more likely than white youths to have two or three of the employment barriers that were analyzed 
(Table 2.14). Recall that, in general, all youths with multiple employment barriers were the most 
likely to receive basic education without training or miscellaneous non-training services, while 
those with none of the employment barrien were the most likely to receive employment-based 
services, including OJT or job search assistance only. 

In general, employment barriers accounted for much of the difference between black and 
white youths in their receipt of basic education without training, and for only some of the 
difference in their receipt of employment-based services (OJT and job search assistance only). 
Table 6.3 displays service receipt rates for white, black, and Hispanic youths within subgroups 



^ote that this difference was also statistically significant among youths with limited work experience 
and cash public assistance receipt. However, only three male youths from this subgroup received job search 
assistance only. As noted above, the discussion focuses on combinations of employment barriers that 
included high school dropout and limited work experience because so few male youths were receiving cash 
public assistance at the time of their JTPA application. 

115 

170 




TABLE 6.3 



SERVICE RECEIPT RATES FOR OUT-OF-SCHOOL YOUTH ENROLLEES, 
BY EMPLOYMENT BARRIER SUBGROUP AND ETHNICITY 



Employment Barrier Subgroup (a) Sample CT-OS (b) 



Non- 
Training JSA 
OJT (c) Education (d) Only 



Misc. 
Non-Training 
Services (e) 



and Ethnicity 


Size 


(*) 




(«) 






(%) 




(«) 




All Employment 






















Barrier Snberotme 






















White, non-Hispanic 


1,138 


29.9 




21.4 




16.0 


13.1 




19.7 




Black, non-Hispanic 


588 


33.0 




7.0 


*** 


31.8 *** 


9.4 


** 


18.9 




HlCMfllO 

xuspaniv 




OU.O 




in o 


*** 


o.y w 






1 A 1 

14.1 




No Employment Barriers 






















White, non-Hispanic 


345 


33.6 




31.3 




4.1 


15.7 




15.4 




Black. non-Hisnanic 


73 


4Q 3 


** 


17 ^ 
i^.j 


*** 




17 R 
1 / .o 








Hispanic 


75 


58.7 




21.3 


* 


1.3 


10.7 




8.0 




1 Employment Barrier 






















Worked fewer than 13 weeks 






















in 12 months before assignment 






















White, non-Hispanic 


148 


35.1 




27.0 




4.1 


12.8 




20.9 




Black, non-Hispanic 


72 


38.9 




9.7 


*** 


9.7 


19.4 




22.2 




Hispanic 


70 


67.1 


### 


15.7 


* 


2.9 


8.6 




5.7 




No high school diploma or GED 






















at assignment 






















White, non-Hispanic 


245 


28.2 




15.5 




19.2 


16.3 




20.8 




Black, non-Hispanic 


102 


23.5 




8.8 




37.3 *** 


9.8 




20.6 




Hispanic 


66 


59.1 


*** 


10.6 




6.1 ** 


6.1 


** 


18.2 




Receiving AFDC or other cash 






















assistance at assignment 






















White, non-Hispanic 


42 


38.1 




23.8 




4.8 


2.4 




31.0 




Black, non-Hispanic 


18 


50.0 




22.2 




0.0 (f) 


22.2 


** 


5.6 


* 


Hispanic 


12 


58.3 




8.3 




0.0 (f) 


33.3 


** 


0.0 


(f) 



2 Employment Barriers 

Worked fewer than 13 weeks 
in 12 months before assignment 
and no high school diploma or GED 
at assignment 

White, non-Hispanic 
Black, non-Hispanic 
Hispanic 



201 
167 
79 



24.9 
19.8 
64.6 



*** 



11.9 
2.4 
6.3 



*** 



31.8 
53.3 
10.1 



*** 
*** 



10.9 
3.6 
0.0 



(f) 



20.4 
21.0 
19.0 



(continued) 
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TABLE 6.3 (continued) 



Non- Misc. 
Training ISA Non-Training 



Employment Barrier Subgroup (a) Sample CT-OS(b) OJT (c) Education (d) Oriy Sendees (e) 
andEtLucity Size (%) (%) <*) (*> 1*L 

Worked fewer than 13 weeks 
in 12 months before assignment 
and receiving AFDC or other cash 

assistance at assignment ion 

37 « ... * ... ' m ; •£ 

2 53 ... « 0.0 (« 9.. 

No high school diploma or GED 
at assignment and receiving 
AFDC or other cash assistance 



*Z£ZL*** 44 22.7 11.4 29.5 15.9 20.5 

BUck, non-Hispanic 25 16.0 8.0 36.0 4.0 36.0 

Hispanic 11 45.5 0.0 (f) 18.2 9.1 27.3 

All 3 Empl"y ment Barriers 

White, n 
Black, ik 
Hispanic 



White, non-Hispanic 74 21.6 5.4 43.2 4.1 25.7 

Black, non-Hisi-nic 92 32.6 3.3 44.6 6.5 13.0 

— 39 46.2 *** 0.0 (f) 23.1 ** 5.1 25.6 

SOURCE: MDRC calculations from Background Information Form responses and from program enrollment 
and participation data from the 16 SDAs. 

NOTES- Calculations for this table are based on data for all white, black, and Hispanic out-of-school youth 
enroUees in the 18-month study sample. EnroUees from other ethnic groups were not included because of small 

sample sizes^ ^ categories in this table are mutually exclusive; see notes b, c, d, and e below. 

For each employment barrier subgroup, within each program service category, a chi-square test 
was applied to the difference in service receipt rates between white enroUees and black enroUees and 
between white enroUees and Hispanic enroUees. Statistical significance levels are indicated as 
* « 10 percent* ** « 5 percent; *** ■= 1 percent. 

(a) The barriers to employment include: not having a high school diploma or GED at the time of 
random assignment; having worked fewer than 13 weeks during the 12 months prior to random assignment; 
and receiving AFDC or other cash assistance at the time of random assignment. 

(b) The CT-OS service category includes enroUees who received CT-OS only, or CT-05> in 
combination with OJT, basic education, job search assistance, or miscellaneous program services. 

(c) The OJT service category includes enroUees who received OJT only, or OJT in combination 
with basic education, job search assistance, or miscellaneous program services. 

(d) The non-training education service category includes enrollees who received basic educauon 
only or basic education in combination with job search assistance or miscellaneous program services. 

(e) The miscellaneous non-training services category includes enrollees who received miscellane- 
ous program services only, or miscellaneous program services in combination with job search »ss,sunce. 
Miscellaneous program services include such activities as assessment, job-readiness preparation, vocational 
exploration, job shadowing, and tryout employment. In this table, miscellaneous program services atao 
S^k experience, which ^identified as a separate program service in other reports on the National 

JTPA Study^ ^ chi _ g<iuare (est was inappropriate because of low expected cell frequencies. 
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defined by each combination of the three employment barriers. 10 The statistical significance levels 
indicate whether the percentage of white youths who received a particular category of services 
differed from the percentage of black or Hispanic youths who received that category of services. 
For example, the second panel shows that among youths with no employment barriers, 34 percent 
of the white youths and 49 percent of the black youths received CT-OS. The table also indicates 
that this difference was statistically significant at the 5 percent level 

Employment barriers appear to account for some of the white/black difference in OJT 
receipt In four of the eight employment barrier subgroups - which included more than 60 
percent of the youths - white youths were more likely to receive OJT than black youths. Also, 
although the top panel in Table 63 shows that there was no overall difference in CT-OS receipt 
rates between white and black youths, there were differences within two of the employment 
barrier subgroups. Among youths with no employment barriers and among those who had limited 
work experience and were receiving public assistance, black youths were much more likely than 
white youths to receive CT-OS and much less likely to receive OJT. It appears that for black 
youths in these two employment barrier subgroups, CT-OS served as a substitute for the OJT, job 
search assistance, and other non-training services that they did not obtain or did not want 
Among youths who were high school dropouts and had limited work experience, black youths 
were less likely than white youths to receive OJT or job search assistance only, but were more 
likely to receive basic education without training. 

Finally, Table 63 indicates that employment barriers accounted partly for the white/Hispanic 
differences in OJT and job search assistance only receipt, but for very litde of the difference in 
CT-OS receipt For example, the differences in OJT receipt rates were not statistically significant 
in three of the employment barrier subgroups and persisted in three others. The difference in 
CT-OS receipt rates persisted in six of the employment barrier subgroups and were not 
statistically significant in the other two. 

Ethnic differences in service receipt may also be due to other factors such as the 
concentration of minority youths in largely urban SDAs or other individual characteristics that may 
be related to ethnicity and service receipt rates. In order to investigate this possibility, the 
analysis was expanded to construct multivariate regression models that estimate service receipt 
rates among ethnic groups while adjusting for differences that might be due to a variety of 



10 Youths from •other - ' ethnic groups were not included in the analysis because of small sample sizes. 
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background and SDA characteristics. The following results are based on a regression model that 
adjusts for differences among ethnic groups associated with age, gender, AFDC history, household 
composition, random-assignment date, Food Stamp receipt at random assignment, previous 
enrollment in job club, having a high school diploma rather than a GED, and SDA characteristics, 
as well as the employment barrier combinations listed in Table 6.3. Like the findings for adults 
discussed in the previous chapter, the results from this model help explain the ethnic differences 
in service receipt rates. For example, the regression-adjusted percentage of white youths who 
received CT-OS was 31 percent compared with 33 percent for black youths, and the regression* 
adjusted percentage for white youths who received OJT was 18 percent compared with 14 percent 
for black youths. For this particular regression model specification, the white/black differences 
in CT-OS and OJT receipt were not judged to be statistically significant. Again, as was true for 
adults, the results from this model indicate that ethnic differences in service receipt can be 
explained on the basis of site differences in service emphasis and differences in service receipt 
among subgroups defined by characteristics other than ethnicity. However, other model 
specifications yielded differences that were statistically significant. 

Again, readers should exercise caution in interpreting the multivariate regression results 
because they are sensitive to the regression model specification used. Also, this type of statistical 
modeling is difficult to translate into policy-relevant and practical conclusions. SDAs must serve 
real people rather than individuals (reflected in the regression model) who are the same on all 
characteristics except ethnicity. Therefore, the analysis shown in Table 63 is probably the most 
realistic picture of ethnic different ; service receipt in the study sites. 

The analysis discussed here (like the analysis in Chapter 5) provides a further context for 
interpreting the ethnic differences in service receipt rates displayed in Table 6.1. Examining these 
differences within employment barrier subgroups and through a multivariate regression framework 
sheds some light on why these differences occurred, but does not account for all of the 
differences. Other factors such as individual characteristics and employment barriers not examined 
here, individual preferences, and SDA, service provider, and employer practices (which, in some 
cases, may amount to racial discrimination) may account for differences in the way JTPA services 
are distributed among ethnic groups. It is also important to keep in mind that previous stages 
in the JTPA client flow also include screening mechanisms that determine service receipt and may 
result in ethnic differences not analyzed in this chapter. 
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IL The Length of Enrollment for Ont-of-School Youths 



A. The Relationship Between Length of Enrollment and Services Received 

like the findings for adults, length of enrollment for youth enrollees was most closely 
associated with the types of services they received. The top row in Table 6.4 lists the median 
length of enrollment for all out-of-school youths and for each of the five service categories. It 
shows that, overall, the median length of enrollment for out-of-school youths was 3.6 months, 
slightly longer than the median length of enrollment for adults (32 months). Table 6.4 also shows 
that the median length of enrollment for those who received CT-OS - the most widely utilized 
service among youths - was 53 months. At the other extreme, the median length of enrollment 
for youths who received job search assistance only was less than one month. In the middle, the 
median length of enrollment for those who received OJT was 3.1 months. This was slightly 
longer than the median length of enrollment for adults who received OJT. Basic education 
without training played a key role in the way that SDAs in the study served out-of-school youths, 
particularly those with multiple employment barriers. Table 6.4 shows that these services were 
relatively long-lasting, with a median length of enrollment of almost four months. 11 

B. Snbgroop Differences in Length of Enrollment 

The relationship between the services that male youth enrollees received and the length 
of their enrollments also helps explain subgroup differences in length of enrollment Table 6.4 
lists selected subgroups of youth enrollees and the median length of enrollment (in months) for 
each subgroup, both overall and by the category of services they received. Statistical tests were 
performed to determine whether there were significant differences in median length of enrollment 
between the first subgroup listed under each characteristic and each of the other subgroups. For 
example, among the subgroups defined by ethnicity, the table indicates that there was a 
statistically significant difference between the median length of enrollment for all white youths 
(3.4 months) and the median length of enrollment for all Hispanic youths (4.8 months). The 
table also indicates that there was no statistically significant difference between the median length 



"These differences in length of enrollment among youths in each of the service categories account for 
similar differences in length of enrollment among youths in each of the recommended service strategies. 
For example, youth enrollees recommended for the classroom training service strategy had the longest 
median length of enrollment at 5.0 months, and those recommended for the OJT/JSA service strategy had 
the shortest median length of enrollment at 2.7 months. Youth enrollees recommended for the other 
services strategy had a median length of enrollment of 3.0 months. 
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of enrollment for white youths and the median length of enrollment for black youths (3.4 
months). 12 

Table 6.4 shows that those who received classroom-based services (CT-OS or basic education 
without training) tended to have longer enrollments than those who received employment-based 
services (OJT or job search assistance only). For example, recall from Table 6.1 that female 
youths were much more likely than male youths to receive CT-OS - the service category with 
the longest median length of enrollment. This accounts for the finding shown in Table 6.4 that 
the median length of enrollment for all female youths was more than half a month longer than 
that for male youths. In tact, except for those who received miscellaneous non-training services 
only, there were no statistically significant differences in median length of enrollment between 
male and female youths who received the same services. 

Table 6.4 also shows that there was no statistically significant difference in the median 
length of enrollment between white and black youths. However, among those who received OJT 
and basic education without training, black youths had shorter enrollments than white youths. 
This suggests that black youths may have been more likely to leave than white youths - either 
for positive reasons such as finding a job or for other reasons such as lack of interest It may 
also be that the training and education that black youths received was oriented toward lower- 
skill occupations and therefore, by design, shorter than that received by white youths. 
Interestingly, Hispanic youths had longer enrollments than white youths among enrollees who 
received CT-OS or OJT. This may be due, in large part, to the fact that Hispanic youths were 
more likely to combine basic education with the occupational training services. 

Among youths who received CT-OS or OJT, high school dropouts had shorter enrollments 
than those who had a high school diploma or GED at the time of their JTPA application. This 
may signify that those who left school previously may have continued this pattern by not 
completing their training or it may be that the training was, by design, shorter than that received 
by graduates and GED recipients. However, the median length of enrollment for high school 
dropouts who received basic education without training was actually slightly longer (although not 
statistically different) than that for graduates who received basic education without training. The 
differences in length of enrollment among other employment barrier subgroups is due, in large 
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Unless otherwise noted, discussion of median length of enrollment differences among subgroups 
focuses on those that are statistically significant at the 10 percent level or lower. 
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part, to differences in the types of services they received. Those who tended to receive basic 
education services - such as those with limited work experience and those receiving cash public 
assistance - had longer enrollments than their more job-ready counterparts, who were more likely 
to receive employment-based services, including OJT or job search assistance only. 
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CHAPTER 7 

VARIATIONS ACROSS THE SERVICE nEMVTOy MWA* 



Chapters 4-6 presented participation information about the entire JTPA study sample, by 
major target group and subgroup. However, by their nature, these pooled, cross-site analyses 
conceal any differences or variations that exist among the 16 diverse SDAs in the sample and, 
therefore, do not permit attempts to determine whether or how local conditions and SDA 
decisions or actions are related to types of services offered or particular types of clients 
enrolled by these 16 SDAs. 

The conceptual framework for examining these kinds of potential relationships at the 
specific SDA level is presented in Figure 7.1. On the left-hand side, under "Local Context of 
the JTPA Program," are listed the three main "external" factors over which the SDAs have little 
or no control but that can reasonably be expected to affect the services offered and the 
population served by a particular SDA labor market and other local economic conditions, 
including the unemployment rate and the degree to which the area is urban or rural; the 
characteristics and size of the JTPA-eligible population residing in the SDA from which actual 
enrollees are drawn; and the broad range of potential service providers in the community, from 
which the SDA selects its actual service providers. On the right-hand side are four areas of 
SDA management and organization over which, as described in Chapter 1, the SDAs can exert 
substantial control, within statutory limits: the nature of the partnership between local elected 
officials and the local private industry council (PIC); the service providers; the SDA's 
administrative processes (particularly recruitment, eligibility, and intake, and the method of 
selecting and contracting with service providers); and the SDA's response to performance 
standards. 1 Together, these external factors (on the left) and areas of SDA discretion (on the 



rr«, ^ da " ficatl0n * "t about thc data for * c > na »ys* of performance standards. 

Fust unlike Chapter 2, which discussed the performance and adjusted performance standards of the 
If study sites for one program year (PY 1988) based on JTPA Annual Status Report (JASR) data 
w°SP! Cr rcUes on *** on P crfonnaDCC and "aj'Kted performance standards that are averaged for" 
each SDA over the three program years of this study (FYs 1987, 1988, and 1989). These data were 
?£t2? mformation P rovidcd ^ «w« " which the SDAs are located, and this procedure was 
followed to permit a fuller view of "performance" over time for the study sites. Typically, there are 

(continued...) 
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FIGURE 7.1 

FACTORS AFFECTING THE IMPLEMENTATION OF JTPA PROGRAMS 



Local Context of the JTPA Program 

• Labor Market Conditions 

• Characteristics and Size of the JTPA- 
Eligible Population 

• Community Employment, Training, 
and Support Services 




Local JTPA Program Management and Organization 

• Nature of the Local Partnership 

• Service Providers 

• Administrative Processes 

• Response to the Performance Standards 



JTPA-Eligibie Population 
Recommended for Services 
(T reatment Group Members) 



JTPA-EligiWe Population 
Enrolled 
(Enroltees) 
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right) lead to the remaining boxes in Figure 7.1: the identification of the JTPA-eligible 
population recommended for service strategies, the population actually enrolled, and the 
particular set of services provided 

Chapter 2 presented and analyzed the high degree of variation across the 16 SDAs in 
local economic conditions and in most of the factors on the right-hand side of Figure 7.1. 
Section II of this chapter examines the variation and patterns of variation among SDAs in the 
services they recommended, the "disadvantagedness" of the population they served, and the 
services JTPA enrollees actually received. This analysis provides important contextual 
information for understanding the pooled, cross-site impacts as well as the impacts of individual 
SDAs presented in the 18-month impact report (Bloom et aL, 1993). Section III then examines 
several commonly held •hypotheses" about the relationship between SDA characteristics and 
the services they are likely to emphasize and the population they are likely to serve. 2 
Examples of the hypotheses based on these potential relationships include the following: 

Possible Hypotheses 

• Areas with higher unemployment rates tend to emphasize classroom training, 
on the grounds that immediate employment opportunities — often viewed as 
a prerequisite for programs emphasizing OJT/JSA - are more limited 

• SDAs that substantially exceed their adjusted performance standards (e.g., job 
placement rates) tend to enroll individuals with fewer employment barriers 
than do SDAs that do not meet, or only marginally meet, their adjusted 
performance standards. 



^...continued) 

few differences in information provided by the states and JASRs. It is true that averaging performance 
over three years disguises the variation within an SDA over the three^^fcriod; about half of the 
SDAs experienced notable variation in performance over these three y^PUSh for adults and youths. 
However, since the study period covered these three years, and since no single year would represent the 
SDAs' performance over the entire study period, the three-year averages appeared to be the most 
appropriate indicator for the desired analysis. 

Second, for the analysis involving adult performance standards, only the entered employment 
rate was used. It was the standard consistently adopted by the study sites and typically drew the most 
attention. The separate analysis involving youth performance standards uses entered employment rate 
and positive termination rate. 

*The analysis in this chapter does not address a third set of issues concerning the relationship 
between SDA characteristics and variation in SDA-specific program impacts. See Bloom et al., 1993, 
for a discussion of this. 
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• In contrast to government-operated SDAs, PIC-operated SDAs tend to 
emphasize OJT as a service, since provides a training subsidy to the private 
sector. (Alternatively, it has been hypothesized that PIC-operated SDAs 
emphasize classroom training to increase the skills level of the potential 
employees,) Further, PIC-operated SDAs also tend to enroll individuals wth 
fewer employment barriers in order to meet the needs of the local employer 
community. 

• SDAs that make greater use of Gxed unit price, performance-based contracts 
enroll larger proportions of less disadvantaged individuals than do SDAs that 
rely mostly on cost-reimbursement contracts. 

It is important to note that these and similar hypotheses are add re ssed far less rigorously 
than are other issues in this report and in the 18-month impact report, for three main reasons. 
First, and most important, these analyses of relationships are nonexperimental; Le., they result 
not from comparing the experiences of treatment and control group members in the same 
location, but from cross-site comparisons in which there are likely to be a host of other 
unmeasured differences that may account for any relationships detected. In addition, relatively 
simple comparative and descriptive methods were used to analyze the data. Hence, the cross- 
site comparisons must be viewed largely as suggestive and descriptive, and not as definitive. 

The second reason is that the number of "observations" - Le., study sites - is quite small 
for this sort of analysis (or for sophisticated statistical modelling techniques, which were not 
undertaken primarily for this reason). Given the large number of possible explanations for 
variations across sites - eg., for the types of individuals served or services provided - and the 
limited number of sites, it is extremely difficult to conclude that one condition (such as the 
unemployment rate or a high job placement rate) is "the" main factor or reason for the 
enrollment of particular types of individuals or the provision of specific services, even if there 
appears to be a strong correlation. 

The third reason concerns the amount and level of available information about the 
external factors - the left-hand side of Figure 7.1 - as well as some of the areas of SDA 



^This caution also applies to any attempts to link local conditions or areas of SDA discretion to 
program impacts. Since both would be nonexperimental cross-site comparisons, they would be less 
reliable; to answer, with equal rigor, the question of how program characteristics affect participation 
patterns, or impacts, would require a "differentiar impact design, in which individuals would be 
randomly assigned to programs having different characteristics. As discussed in Chapter 3, measuring 
differential impacts was not a goal of the National JTPA Study. For a full discussion of the difficulties 
inherent in drawing conclusions from cross-site comparisons, see Cave and Doolittle, 1991, pp. 161ff. 

129 



discretion that further limit the comprehensiveness and depth of analyses presented in this 
chapter. As stated in Chapter 1, the Request for Proposals (REP) issued by DOL did not 
envision the collection of a large amount of information on the participating SDAs beyond the 
type of enrollment and participation data and information on administrative structures 
presented in the preceding chapters* Only a small amount of information about the external 
factors was collected on the participating SDAs, such as the basic unemployment rate, wage 
level, and population data presented in Chapter Z Apart from a forthcoming nonexperimental 
study conducted in four of the SDAs, little information was collected on the characteristics of 
the JTPA-cligible population or on the range of potential service providers. Hence, regarding 
the topics listed on the right-hand side of Figure 7.1, analysis must be quite limited. 

With these important cautions in mind, this chapter now shifts the focus from ihe entire 
"pooled" sample of treatment group members analyzed in the preceding four chapters to site* 
specific findings in the 16 individual SDAs that agreed to be part of the National JTPA Study. 

I. An Overview of the Findings 

Several findings emerge from this chapter: 

• SDAs varied widely in the types of services they recommended for adults and 
youths. For example, the proportion of adults recommended for the 
OJT/JS A service strategy ranged from 7 percent to 79 percent; the proportion 
of youths recommended for the other services strategy ranged from 0 percent 
to 88 percent, 

• Similarly, SDAs varied widely in the types of services that enrollees actually 
received. For example, the proportion of adult enrollees who received basic 
education not in combination with OJT or classroom training ranged from 0 
percent to 47 percent; for youth enrollees, it ranged from 0 percent to 80 
percent 

• With respect to treatment group members who had multiple employment 
barri< ?, there were large differences across sites in the formal JTPA 
enrol nent rate. 4 For example, for adults with multiple barriers, the 



4 As discussed in Chapter 5, the employment barriers used in this study were based on amount of 
prior employment, educational attainment, and welfare receipt Other objective and subjective 
characteristics that many experts believe to be important indicators of disadvantage are not considered 
in these analyses. 
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enrollment rate ranged from 24 percent to 91 percent, and for youths it 
ranged from 21 percent to 88 percent 

• Two SDAs had substantially higher formal JTPA enrollment rates for adults 
with multiple employment barriers than for adults with none of the 
employment barriers studied For youths, this was true in three SDAs. 
However, four SDAs had considerably lower enrollment rates for adults with 
multiple employment barriers than for adults with none of the employment 
barriers studied For youths, this was true in only one SDA. (Two caveats: 
These findings may reflect the characteristics of the pool of eligible 
individuals in the individual SDAs, over which the SDAs have no control; or 
they may result from the level of recruitment activities or selection/screening 
processes undertaken by the SDAs. 5 ) 

• There is some very preliminary evidence that SDAs that substantially 
exceeded their adult performance standards, compared with those that 
marginally exceeded their standards, placed greater emphasis on O JT/IS A - 
and less emphasis on classroom training — for adults, and tended to enroll 
a less disadvantaged adult population. (The caveats noted above also apply 
to these findings*) However, a different and more mixed pattern emerged 
for youths and depended on the performance standard used to characterize 
the SDAs. For example, SDAs that marginally exceeded the youth entered 
employment standard placed especially heavy emphasis on classroom training, 
while SDAs that marginally exceeded the youth positive termination standard 
recommended the three service strategics in roughly equal proportions. 
However, SDAs that substantially exceeded either of the two youth standards 
did place greatest emphasis on the other services strategy and tended to 
enroll a more disadvantaged youth population than did SDAs that marginally 
exceeded their standards. 6 

• Analyses did not reveal any clear or consistent relationships among examined 
structural and administrative characteristics of the 16 SDAs. For example, 
compared with government-operated SDAs, PIC-operated SDAs did not have 
consistently higher or lower performance. High use of fixed unit price 



5 It can be assumed that some form of selection or screening occurred in most SDAs - on the 
part of both applicants and staff - prior to applicants 9 reaching the point of being recommended for 
a service strategy. The variations in formal enrollment patterns discussed in this chapter reflect 
selections that took place after applicants were determined eligible for JTPA, recommended for services, 
and randomly assigned to the treatment group. 

*The different pattern for adults vis-i-vis youths is difficult to explain. One possible explanation 
may be that performance standards for youths included 'positive termination" outcomes such as 
completion of schooling or attainment of employment competencies, not just (as with adults) standards 
related to job placement. This could have permitted, or encouraged, SDAs to enroll a more 
disadvantaged youth population - for whom job placement at termination from JTPA was not seen to 
be an appropriate goal - without jeopardizing their ability to meet performance standards. 
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contracts did not appear to be associated with higher levels of performance, 
or with PIC-operated SDAs compared with government-operated SDAs. 

• Also, compared with government-operated SDAs, PIC-operated SDAs did not 
seem to enroll greater or lesser proportions of more disadvantaged people 
than of less disadvantaged people. High use of fixed unit price contracts did 
not seem to be correlated with the enrollment of greater proportions of less 
disadvantaged than more disadvantaged people. Finally, unemployment rates 
did not seem to be related to recommended service strategies or the 
distribution of actual services received 

The remainder of this chapter discusses these and other findings in more detail 

n. Variation fa Service Strategy Recommendations. Popnlatton Served. 
Mid Services Received 

As noted in Chapter 1, within broad statutory requirements such as program eligibility, 
the JTPA statute explicitly gives authority to SDAs to determine the population served and the 
services provided. 7 The legislation anticipated that different local conditions and decisions 
would lead to the recommendation and provision of different services across the country to 
different types of individuals. Not unexpectedly, there was substantial variation across the 16 
SDAs in these areas. 

A* Service Strategy Recommendations 

As noted in Chapter 3, SDA staff developed service recommendations for JTPA 
applicants based on a combination of several factors, including an employability assessment, the 
client's interests and preferences, the availability and eligibility of appropriate service providers 
in the community, and the perceived likelihood that the client would be accepted by the 
service provider. The recommendations that resulted from these factors reflect the emphasis 
and priorities SDAs gave to the various service strategies. 

Table 7.1 presents the percentage distributions of service strategy recommendations 
among adult and youth treatment group members. In an effort to highlight the variation 
among the sites, the table ranks the SDAs by their degree of emphasis on the OJT/JSA service 
strategy. The left panel of Table 7.1 shows that five SDAs recommended OJT/JSA for more 
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than half of the adults in the treatment group (Decatur, Springfield, Northwest Minnesota, 
Fort Wayne, and Corpus Christi), three SDAs recommended classroom training for more than 
half of the adults in the treatment group (Omaha, Cedar Rapids, and Jackson), and two SDAs 
recommended the other services strategy for more than half of the adults in the treatment 
group (Larimer and Providence). The right panel of Table 7.1 shows that there is some 
overlap among SDAs that emphasized the same service strategies for both adults and youths. 
For example, Decatur, Northwest Minnesota, and Fort Wayne also emphasized OJT/JSA for 
youths; Cedar Rapids and Omaha also emphasized classroom training for youths; and Larimer 
and Providence also emphasized other services for youths. 

Finally, three SDAs appear to have operated distinctly different programs for adults and 
youths. Jackson emphasized classroom training for adults but placed a heavy emphasis on 
other services for youths. Corpus Christi emphasized OJT/JSA and, to a lesser extent, 
classroom tanning for adults, while placing heavy emphasis on classroom training for youths. 
Springfield placed a very he^vy emphasis on OJT/JSA and very little emphasis on other services 
for adults, but was much more likely to recommend other services and much less likely to 
recommend classroom training for youths. 

B. Population SgrwH 

JTPA was designed to provide job training for economically disadvantaged individuals and 
others facing serious barriers to employment However, because access to JTPA is not 
guaranteed to all who satisfy the eligibility rules, the characteristics of the population enrolled 
in JTPA is a function of a variety of factors, deluding the characteristics of the eligible 
population in the SDA the targeting and intensity of recruitment, and the types of services 
available to and desired by the eligible population. In addition, SDA staff and service 
providers, as well as the clients themselves, can exercise discretion over whether clients enroll 
after having applied and having been recommended for services. Given the variability in these 
factors across the country, it is reasonable to expect great variation across SDAs in the 
characteristics of JTPA enrollees. However, it is impossible to attribute the variation to any 
one of these factors. 

A full analysis of the differences across SDAs in the extent to which they served more 
or less disadvantaged clients would at least require information on the enrollment rates for the 
eligible populations in each SDA but these data were not collected as part of the 
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implementation study. For example, it is not possible to determine the degree to which 
differences in the pool of those recommended for services reflect other factors: differing 
characteristics of the eligible populations across the 16 SDAs, variations in the level of 
recruitment activities undertaken by SDAs (e.g., "passive" recruitment vs. aggressive efforts to 
identify and attract more disadvantaged individuals), or differences in the pre-random- 
assignment selection and "matching" process used by SDAs and clients. In the absence of this 
information, the analysis must rely on comparisons that can be made using available data from 
later stages in the selection and enrollment process, which means that any findings are, at most, 
suggestive and not definitive. The present discussion examines variation across SDAs in the 
extent to which they formally enrolled more or less disadvantaged treatment group members 
as defined by the three key employment barriers used earlier in this report 

Tables 12 and 73 present the percentage of adults and youths, respectively, in two of 
the three employment barrier subgroups defined in Chapters 2-6: those with two or three of 
the employment barriers and those with none of the employment barriers. In examining the 
percentage of adults with two or three employment barriers, it appear? that some SDAs 
targeted larger proportions of more disadvantaged clients than did other SDAs. For example, 
Table 12 indicates that 12 percent of the adults in Larimer's treatment group had two or more 
employment barriers compared to 47 percent in Marion. Among youths, the variation was 
almost as wide: 21 percent of Larimer's youth treatment group members had two or more 
employment barriers compared to 52 percent of those in Jackson. Again, however, it is not 
clear whether this was due to the characteristics of the eligible population, to the SDAs' 
recruitment activities, or to the decisions of service providers, SDA staff, or the clients 
themselves. 

Tables 12 and 7.3 also highlight the great variation in the enrollment rates for treatment 
group members in the employment barrier subgroups. At first glance, it appears that some 
SDAs were more successful than others in targeting the more disadvantaged for enrollment. 
For example, Table 7.2 shows that Northwest Minnesota enrolled 24 percent of the adult 
treatment group members with two or three employment barriers compared with an enrollment 
rate of 91 percent for the same group in Larimer. Table 73 shows similar variation in 
enrollment rates among youths with multiple employment barriers. However, these differences 
do not take into account the fact that some SDAs with low enrollment rates among the most 
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disadvantaged also had low enrollment rates among the least disadvantaged Similarly, some 
SDAs had high enrollment rates among both groups. Similar enrollment rates among both 
groups suggests that there was no c!ear enrollment targeting for the more disadvantaged in 
these SDAs. 

An alternative approach provides an opportunity to examine the extent to which 
enrollment rates differed among more and less disadvantaged treatment group members in the 
same SDA. This approach compares the enrollment rate for more disadvantaged individuals 
(those with two or three employment barriers) with the enrollment rate for less disadvantaged 
individuals (those with no employment barriers). Tables 12 and 73 rank the SDAs by these 
differences, which are displayed in the last column of each table. A positive difference means 
that the SDA enrolled a greater proportion of its more disadvantaged treatment group pool 
than of its less disadvantaged treatment group pooL A negative difference means the reverse. 

It is useful to Valk through" an example. Table 12 shows that Omaha enrolled 56.8 
percent of the adult treatment group members who had two or three employment barriers and 
40.6 percent of those who had no employment barriers. Hence, the difference is 16.2 
percentage points (56.8 minus 40.6), indicating that Omaha enrolled a larger proportion of 
more disadvantaged adults (from its recommended pool) than less disadvantaged adults. The 
corresponding difference for Northwest Minnesota is -40.9 percentage points, indicating that 
it enrolled a larger proportion of less disadvantaged adults (from its recommended pool) than 
more disadvantaged adults. It is also interesting to note that Omaha and Northwest Minnesota 
had similar proportions of disadvantaged adults with multiple employment barriers in their 
recommended pools (33.0 percent compared with 34,9 percent, respectively). 

This approach takes into account differences across SDAs in the distribution of treatment 
group members among the employment barrier subgroups. It also takes into account the fact 
that some SDAs had high enrollment rates for all groups, while others had high enrollment 
rates for some groups and low enrollment rates for others. Note that this approach still cannot 
take into account the recruitment or screening process that took place before applicants 
became part of the recommended pool. 

An examination of the last column in Tables 12 and 7.3 reveals several points: 

• There was great variation across sites in the difference in enrollment rates 
for those with multiple employment barriers and those with none. (This 
variation is greater than the sizable cross-site variation in the proportion of 
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ttcatment group members with multiple bakers, which is shown in column 
2 of Tables 12 and 73.) For example, in Jackson, the enrollment rate for 
youths with two or three employment barriers was 61 percentage points 
higher than it was for youths with none of the barriers, whereas in Northwest 
Minnesota, the rate for youths with two or three barriers was 41 percentage 
points less than it was for youths with none. 

• Omaha enrolled substantially greater proportions of adults with multiple 
employment barriers (compared with the rate for those with none) from 
among the pool of those recommended for services. Among youths, this 
was true of Jackson, Omaha, and Marion. 

• Three SDAs enrolled substantially larger proportions of adults with none of 
the employment barriers (compared with those with multiple barriers) from 
among the pool of those they recommended for services: Northwest 
Minnesota, Springfield, and Fort Wayne. Among youths, this was the case 
only m Northwest Minnesota. 

• The differences in enrollment rates between the two employment barrier 
subgroups do not appear to have been associated with the proportion of 
treatment group members with multiple employment barriers. For example 
Northwest Minnesota and Omaha had approximately the same percentage of 
adults with two or three barriers. However, Omaha enrolled a much larger 
proportion of those with multiple barriers than it did those with no barriers 
THe patterns for Northwest Minnesota ran in the opposite direction Also' 
SDAs with similar enrollment rate differences among the employment barrier 
subgroups had very different proportions of adults or youths in those groups. 

Comparing the rankings in Tables 12 and 73 suggests that most SDAs had roughly 
similar enrollment rate differences among youths and adults in the same employment barrier 
subgroups. However, three SDAs stand out as having had distinct enrollment rate differences 
for adults and youths in the same employment barrier subgroups. In Fort Wayne and 
Springfield, adults with multiple employment barriers were less likely to enroll than adults with 
no employment barriers, while the opposite was true for youths. In Jackson, adults from both 
employment barrier subgroups had roughly the same enrollment rates, while youths with 
multiple employment barriers were much more likely to enroll than were those with no 
barriers. 
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C Services Received 



Table 7.4 presents the service receipt rates for adults and youths who were enrolled in 
JTPA during the follow-up period. (Following the presentations in Chapters 5 and 6, the 
service categories in this table are mutually exclusive.) In order to highlight the wide variation 
in service emphasis, the SDAs are ranked by the percentage of enrollees who received OIL 
Once again, the variation in service emphasis is quite substantial There are a number of 
findings of interest: 

• Two SDAs placed a clear emphasis on OJTs for both adults and youths: in 
both Cedar Rapids and Northwest Minnesota, more than half of both the 
adult and the youth enrollees received OJTs. In four SDAs - Omaha, 
Jackson, Jersey City, and Oakland - more than half of the adult enrollees 
received classroom training, while in only Corpus Christi and Omaha did 
more than half of the youth enrollees receive classroom training. In 
Providence and Larimer, more than half of the adults and youths received 
miscellaneous non-training services. 

• There was wide variation in the receipt of job search assistance as a single 
service (ranging from 0 percent to 47 percent for adults and from 0 percent 
to 37 percent for youths). 

• For four SDAs, basic education was not provided, at least as a discrete 
service, for either adult or youth enrcllees. However, as many as 47 percent 
of the adult enrollees in Marion and 80 percent of the youth enrollees in 
Jackson received basic education without training. 

• Most SDAs emphasized similar services for adults and youths, but three 
SDAs stand out in emphasizing different services for each group. In Jackson, 
more than 7 of 10 adult enrollees received classroom training, while 8 of 10 
youth enrollees received basic education without training. In Corpus Christi, 
69 percent of the youths received classroom training compared to 35 percent 
of the adults, while 25 percent of the adults received basic education without 
Uaining compared to 9 percent of the youths. In Heartland, 36 percent of 
the youth enrollees received miscellaneous non-training services, and only 29 
percent received classroom training. In contrast, only 8 percent of the adult 
enrollees in Heartland received miscellaneous non-training services, while 44 
percent received classroom training. 

A comparison of Tables 7.1 and 7.4 indicates that, with the exception of two SDAs, 
servk e emphasis, as reflected in recommended service strategies, was roughly similar to service 
emphasis reflected in actual services received. For example, Decatur, Springfield, Northwest 
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8.0 
0.0 
0.0 
79.7 
13.6 
39.2 
n/a 


VC 
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Youths 
CT-OS (b) 


31.6 
18.4 

41.7 
17.6 
68.5 
32.2 
20.0 
28.9 
48.6 
68.4 
33.3 
17.1 
24.2 
12.2 
n/a 
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65.8 
57 1 

•J I * 1 

29.0 
20.5 
19.6 
16.7 
16.1 
15.0 
8.9 
8.7 
7.0 
1.6 
1.4 
0.0 
0.0 
n/a 


15.5 


CM 

m 
m 


Site 


Northwest Minnesota 
Cedar Rapids, Iowa 
Fort Wayne, Ind. 
Coosa Valley, Ga. 
Springfield, Mo. 
Corpus Christi, Tex. 
Butte, Mont. 
Decatur, III. 
Heartland, Fla. 
Jersey City, N. 
Omaha, Neb. 
Providence, R.I. 
Jackson, Miss. 
Larimer County, Colo. 
Marion, Ohio 
Oakland, Calif, (e) 


All Sites 


Sample Size 


Misc. 
Non-Tr. 
Serv. (d) 
(%) 


13.3 
17.8 
16.7 
23.3 
1.5 
2.6 
20.6 
4.0 
7.8 
0.2 
14.2 
0.9 
10.0 
62.2 
0.7 
50.9 


14.0 


VO 
CM 


Non-Tr. 
Ed. (c) 
(%) 


11.9 
0.0 
1.0 
3.0 
0.0 
0.6 
19.1 
0.0 
1.0 
24.9 
0.0 
3.6 
47.4 
0.0 
0.0 
7.5 




O 


Adults 

CT-OS (b) 
(%) 


20.7 
29.0 
46.0 
13.4 
16.5 
24.1 
37.1 
72.8 
44.1 
35.1 
75.2 
59.1 
17.7 
31.2 
58.5 
33.5 


38.0 


1,964 


< H 
co c 
— > O w | 


0.0 
2.8 
1.0 
25.2 
47.4 
40.6 
0.0 
2.6 
29.4 
24.3 
0.0 
27.7 
17.9 
0.0 
36.7 
4.3 


19.2 


to 

ON 


0 


54.1 
50.5 

35.1 
34.6 
32.2 
23.2 
20.7 
17.6 
15.4 
10.6 
8.6 
6.9 
6.6 
4.1 
3.7 


20.8 


1,077 


Site 


Cedar Rapids, Iowa 
Northwest Minnesota 
Coosa Valley, Ga. 
Decatur, III. 
Fort Wayne, Ind. 
Springfield, Mo. 
Butte, Mont. 
Jackson, Miss. 
Heartland, Fla. 
Corpus Christi, Tex. 
Omaha, N^b. 
Jersey City, N.J. 
Marion, Ohio 
Providence, LI. 
Oakland, Calif. 
Larimer County, Colo. 


All Sites 


Sample Size 
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Minnesota, and Fort Wayne recommended the OJT/JSA strategy for well over half of the 
adults, and more than half of the adult enroUees in these SDAs received either OJT or JSA* 

m ' BctWCCa m rharactgrisfi " ^ H S gda Emcfeasis snjj 

Section H described the variations among the 16 SDAs in the types of services for which 
clients were recommended, the population that was ultimately enrolled, and the types of 
services enroUees received Section m addresses the question of whether these variations 
appear to have been associated with the variations in selected SDA characteristics that were 
presented in Chapter 7? As noted earlier, this discussion will examine a number of 
"hypotheses" about these relationships. 

A- The Relationships Among SDA Qaactgjgggg 

Before moving on to the analysis of these relationships, it should be noted that there 
do not appear to be clear or consistent relationships among the SDA characteristics examined 
here. Following is a brief summary of findings from the analysis of relationships among the 
JTPA program management and organizational characteristics described earlier: the nature of 
the local partnership; the extent to which the SDAs used fixed unit price, performance-based 
vendor contracts; and the level of performance on entered employment rate standards for 
adults and youths and positive termination rate for youths. 

L SPA organizational structure (nature of the loc a l partnership „ n d type of v»nHnr 
fimlracts. One hypothesis is that PIC-operated SDAs would tend to use fixed unit price, 
performance-based contracts extensively, on the grounds that the private sector is more 
oriented toward "bottom-line" results. (Such contracts withhold full payment for services until 
a speciGed outcome is achieved.) In fact, during the development of the JTPA legislation, 



in c2 p2 ^ f recommended and actual services was quite different for adults and youths 
in Cedar Rapids and for youths m Decatur. In Cedar Rapids, a high percentage of adults and vouths 
were ^recommended for the classroom training service strategy, but mSre than jjtf of Z aduteanl 

for OJT0SA, and 5 percent were recommended for the other services strategy, but only 25 percent of 
enrollees received either OJT or JSA, and 48 percent received miscellaneoJ Ti^ttSnfseSc^ I 
was not possible to identify a plausible explanation for the difference between recom3ed a^dTctua 
services based on site observations during the study period. ana actuai 

*The reader is again reminded of the cautions that appear at the beginning of this chapter 
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fixed unit price contracts were often referred to as the major way the private sector purchases 
services. The corollary hypothesis is that government-operated SDAs would make little or no 
use of fixed unit price contracts and more use of cost-reimbursement contracts, which assure 
full payment regardless of outcome. 

Comparing Table 23 with Table 2.6 provides little support for these hypotheses. It is 
true that, of the six government-operated SDAs (see Table 23), four made little or no use 
of fixed unit price contracts (see Table 2.6): Butte, Cedar Rapids, Decatur, and Springfield 
The other two, Heartland and Omaha, made only moderate use of them. But no pattern 
emerges with the PICoperated SDAs. Of the three in that category (see Table 23), two made 
little or no use of fixed unit price contracts, and one made high use. Of the three SDAs that 
had the highest use of fixed unit price contracts for both adults and youths - Coosa Valley, 
Northwest Minnesota, and Jackson - two were jointly operated by the PIC and government, 
and one (Northwest Minnesota) was PICoperated. 

2. Type of vendor contracts and SPA performance . As noted above, full payment 
under fixed unit price contracts is withheld until certain performance measures are met - i.e M 
for adults, placement in a job at a certain wage rate for a specific period of time, and for 
youths, these or other positive termination outcomes, such as completion of a GED. One 
obvious hypothesis is that SDAs that use fixed unit price contracts achieve higher rates of 
adjusted performance - as defined by the degree to which they exceeded their adjusted 
entered employment rates (for adults) and entered employment and positive termination rates 
(for youths) - than do SDAs that use cost-reimbursement contracts. 10 

There is little evidence to support this hypothesis for either adults or youths. Of the 
three SDAs that made the highest use of fixed unit price contracts for adult training programs 
(see Table 2.6), two substantially exceeded their standards and one marginally exceeded its 
standards. But of the eight SDAs that made little or no use of fixed unit price contracts for 
adults, three substantially exceeded and two moderately exceeded their standards. Similarly, for 
youths, among the five SDAs that made the greatest use of fixed unit price contracts, two 
substantially exceeded and two marginally exceeded their entered employment rate standards; 



10 Here and later in this chapter, SDAs are broken out into three groups: SDAs that substantially 
exceeded the performance standard (by 15 percent or more), those that moderately exceeded the 
standard (by 10 to 14.9 percent), and those that did not meet the standard or marginally exceeded it 
(by less than 10 percent). 
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a similar story presents itself for youth positive termination rates. 

Hence, these 16 SDAs do not provide justification for the view that u * of fixed unit 
price contracts is associated with higher performance than use of cost-reimbursement contracts. 

3- SPA SnailteflgBal structure (nature of the local partnership »H gp ^ 
performance. Another hypothesis is that, owing to the stronger private sector role in programs 
run by PICs, performance will be better in those programs than in government-operated 
programs. A comparison of each SDA's administrative arrangement and performance levels 
shows no support for this hypothecs. Of the two SDAs in the study that substantially 
exceeded all three performance measures analyzed - adult entered employment rate, youth 
entered employment rate, and youth positive termination rate - one (Jackson) had a shared 
PIC/government-operated structure and the other (Springfield) was government-operated. 
Further, among the three PlOoperated SDAs shown in Table 23, there is no consistent 
pattern in their degree of performance: for adults, one substantially exceeded its performance 
standards, and two marginally exceeded their performance standards. For youths, all three PIC- 
operated SDAs marginally exceeded the youth performance standards. Therefore, these 16 
SDAs do not appear to provide support for this hypothesis. 

B. SPA Chara cteristics and Service Emphasis 

This discussion focuses on three SDA characteristics - the unemployment rate (see Table 
22), the extent to which SDAs exceeded their performance standards, and the nature of the 
local partnership (see Table 23) - and their possible relationship to the emphasis SDAs 
placed on service strategy recommendations (see Table 7.1) and service receipt (see Table 7.4). 
Although other SDA characteristics were examined, these three are presented for discussion 
because they are particularly compelling conceptually and, in the case of SDA performance and 
organizational structure, because there appears to be suggestive evidence that they are related 
to an SDA's service emphasis, at least in the 16 study sites. 11 

L The unemployment rate and service emp hasis it is often assumed that the level 



Two other SDA characteristics that may be associated with service emphasis were also examined- 
urban/rural distinctions (see Table 2.2) and the extent to which SDAs used fixed um? price 
performance-based vendor contracts (see Table 26). Neither of these characteristics appears to have 
been associated with either service strategy recommendations or service receipt patterns m the 16 SDAs 
in the study. 
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of unemployment in an SDA is a major determinant of the types of services clients demand 
and SDAs are likely to provide. One hypothesis is that more classroom skills training would 
be offered in areas with a higher unemployment rate, because immediate employment 
opportunities, often seen as a prerequisite for OJT and job search assistance, are more limited 
JTPA applicants may also prefer classroom training as a means to retool the skills they used 
in previous jobs so that they can enter new occupations. 

Interestingly, the unemployment rate in an SDA does not seem to have been consistently 
related to emphasis on a particular service strategy recommendation or to the distribution of 
actual services received among the 16 study sites. The three sites with the lowest 
unemployment rates (see Table 22) - Cedar Rapids, Omaha, and Providence - recommended 
(Table 7.1) and provided (Table 7.4) very different mixes of OJT, classroom training, and other 
services for both adults and youths. Similarly, no consistent pattern can be seen in the service 
strategies of the SDAs with the highest unemployment rates - Corpus Christi, Decatur, and 
Heartland. 

2. SDA performance and service emphasis . One widely held view in the JTPA 
system is that higher levels of performance among SDAs — as defined by the degree to which 
they exceeded their adjusted entered employment rates (for adults) and entered employment 
and positive termination rates (for youths) — are associated with an emphasis on specific direct 
employment strategies (i.e., OJT or job search assistance rather than classroom training). The 
reasoning suggests that, in SDAs that strive to exceed performance standards, particularly since 
many of the performance standards (especially for adults) are tied closely to labor market 
outcomes, JTPA applicants may be encouraged to use services that get them attached to the 
labor market as soon as possible. It is also possible that the greater focus on positive 
terminations for youths (including completion of schooling or attainment of employment 
competencies) may encourage SDAs to recommend more classroom training and basic 
education for youths. 

Tables 7.5 and 7.6 present information that permits a preliminary assessment, using 
entered employment rate as the performance measure for both aJults and youths in Table 15 
and positive termination rate only for youths in Table 7.6. The percentages of adults and 
youths recommended for each of the three service strategies are broken out for three groups 
of SDAs: SDAs that substantially exceeded the performance standard (by 15 percent or more), 
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TABLE 7.5 



PERCENTAGE DISTRIBUTIONS OF SERVICE STRATEGY RECOMMENDATIONS 
FOR ADULT AND YOUTH TREATMENT GROUP MEMBERS, 
BY SITE ACHIEVEMENT OF THE JTPA PERFORMANCE STANDARD 
FOR ENTERED EMPLOYMENT 

Site Achievement of ~~~ 



Performance Standard Adults Youths 

and Service Strategy Recommendation (%) f *\ 



Sites that Substantially 
Exceeded the Standard (*\ 






Classroom Training 
OJT/JSA 
Other Services 
Total 


27.6 
52.6 
19.8 
100.0 


30.2 
27.4 
42.4 
100.0 


Sites Oat Moderately 
Exceeded the Standard (h) 






Classroom Training 
OJT/JSA 
Other Services 
Total 


28.1 
50 0 

21.9 
100.0 


50.1 
24.9 
24.9 
99.9 


Sites that Marginally 
Exceeded the Standard (c\ 






Classroom Training 
OJT/JSA 
Other Services 
Total 


47.3 
24.3 
28.4 
100.0 


61.2 
29.8 
9.1 
100.1 


Difference (d\ 






Classroom Training 
OJT/JSA 
Other Services 


-19.7 
28.3 
-8.6 


-31.0 
-2.4 
33.3 


SOURCE: MDRC calculations from Background Information Form responses and from performance 
standard and actual performance data from the 16 SDAs. See Appendix C for information about the 
performance standard data. 


NOTES: Calculations for this table are based on data for all treatment 
study sample. 


group members in the 18-month 



Distributions may not total 100.0 percent because of rounding. 
Tests of statistical significance were not performed. 

(a) This subgroup includes sites that exceeded the entered employment rate performance standard 
by IS percent or more. 



(b) This subgroup includes sites that exceeded the entered employment rate performance standard 
by 10 to 14.9 percent. 

(c) This subgroup includes sites that exceeded the entered employment rate performance standard 
by less than 10 percent. (Some sites included in this subgroup did not exceed the performance standard.) 

(d) This is the recommendation rate difference between sites that substant-illy exceeded the 
standard and those that marginally exceeded it. 
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TABLE 7.6 



PERCENTAGE DISTRIBUTIONS OF SERVICE STRATEGY RECOMMENDATIONS 
FOR YOUTH TREATMENT GROUP MEMBERS, 
BY SITE ACHIEVEMENT OF THE JTPA PERFORMANCE STANDARD 
FOR YOUTH POSITIVE TERMINATIONS 



Site Achievement of 

Performance Standard Youths 

and Service Strategy Recommendation (%) 

Sites flat Substantially Exceeded the Standard fe) 

Classroom Training 26.2 

OJT/JSA 2L1 

Other Services 52.7 

Total lOO.o 

Sites that Moderately Exceeded the Standard (b) 

Classroom Training 56.4 

OJT/JSA 2i\ 3 

Other Services 12.4 

Totol 100.1 

Sites that Marginally Exceeded the Standard (cl 

Classroom Training 36.7 

OJT/JSA 30^2 

Other Services 33.1 

Totai 100.0 



Difference (d) 

Classroom Training -10.5 

OJT/JSA -9.1 

Other Services 19.6 



SOURCE: MDRC calculations from Background Information Form responses and from 
performance standard and actual performance data from the 16 SDAs. See Appendix C for 
information about the performance standard data. 

NOTES: Calculations for this table are based on data for all youth treatment group members 
in the 18-month study sample. 

Distributions may not total 100.0 percent because of rounding. 

Tests of statistical significance were not performed. 

(a) This subgroup includes sites that exceeded the youth positive termination rate 
performance standard by IS percent or more. 

(b) This subgroup includes sites that exceeded the youth positive termination rate 
performance standard by 10 to 14.9 percent. 

(c) This subgroup includes sites that exceeded the youth positive termination rate 
performance standard by less than 10 percent. (Some sites included in this subgroup did not 
exceed the performance standard.) 

(d) This is the recommendation rate difference between sites that substantially 
exceeded the standard and those that marginally exceeded it. 
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SDAs that moderately exceeded the performance standard (by 10 to 14.9 percent), and SDAs 
that did not meet the performance standard or that marginally exceeded it (by less than 10 
percent). The bottom panel of each table presents the difference in service emphasis between 
the SDAs that substantially exceeded the standard and the SDAs that marginally exceeded the 
standard; for example, a positive difference means that SDAs that substantially exceeded the 
standard placed greater emphasis on the indicated activity compared to those that marginaly 
exceeded the standard. A negat /e difference would mean the opposite. 

In general, it appears that, for adults, SDAs that substantially exceeded their performance 
standards tended to emphasize the OJT/JSA service strategy when compared to SDAs that 
marginally exceeded their standards. The top panel of Table 7.5 shows that SDAs that 
substantially exceeded their entered employment rate standard recommended the OJT/TSA 
service strategy for adults much more frequently than either the classroom training or other 
services strategy. In feet, these SDAs recommended OJT/JSA more than twice as often for 
adults as SDAs that only marginally exceeded their entered employment rate performance 
standards - 52.6 percent vs. 243 percent, for a 283 percentage point difference, as shown in 
the bottom panel of Table 7.5. The table also shows that, for both youths and adults, SDAs 
that substantially exceeded their performance standard were much less likely to recommend the 
classroom training service strategy than were SDAs that marginally exceeded their performance 
standards (a -20 percentage point difference for adults and a -31 percentage point difference 
for youths). Table 7.6 shows the same pattern of lower rates of classroom training 
recommendations for youths among SDAs that substantially exceeded their positive termination 
performance standard (a -10.5 percentage point difference). 

Interestingly, for youths, SDAs that substantially exceeded either their entered 
employment rate or their positive termination rate performance standards were much more 
likely to have recommended the other services strategy than were SDAs that marginally 
exceeded these performance standards (a 33 percentage point difference for entered 
employment rate and a 20 percentage point difference for positive termination rate). Since this 
activity category for youths consists largely of basic education and other classroom-based 
activities, the hypothesis that SDAs that substantially exceeded performanu. standards would 
use less classroom training and more OJT/JSA does not have much support for youths in these 
16 SDAs - although it does have substantial support for the adults. 
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3. SPA organizational structure (nature of the local partnership) and service 
emphasis . It is often suggested that one result of strong private sector involvement in JTPA 
would be an emphasis on providing services, such as OJT, that offer the equivalent of a 
training subsidy to business. (The opposite hypothesis - that PIC-operated SDAs would favor 



adults and youths who were recommended for classroom training, OJT/JSA, and "other" 
services. These percentages are broken out for the three groups of SDAs presented in Table 
23: SDAs operated predominantly by a government entity, SDAs operated by a combination 
of a government and the PIC, and SDAs operated predominantly by the PIC The bottom 
panel presents the difference in service strategy recommendations between the government- 
operated and PIC-operated SDAs; for example, a positive difference means that the 
government-operated SDAs placed greater emphasis on the indicated service strategy compared 
to PIC-operated SDAs. A negative difference would mean the opposite. 

Table 7.7 indicates that, for both adults and youths, PIC-operated SDAs tended to 
recommend the OJT/JSA service strategy much more frequently than either the classroom 
training or the other services strategy. Further, PIC-operated SDAs recommended OJT/JSA 
more frequently than did the government-operated SDAs (as shown by the negative value in 
the "OJT/JSA" row in the bottom panel for both adults and youths). Although government- 
operated SDAs recommended OJT/JSA most frequently for adults, they recommended 
classroom training for both adults and youths much more frequently than did PIC-operated 
SDAs. Overall, therefore, there 1% at best weak support for the hypothesis that strong private 
sector involvement in JTPA is associated with greater use of OJT services. 

C SPA Characteristics and Population Served 

This discussion focuses on three SDA characteristics - the extent to which SDAs 
exceeded their performance standards, the nature of the local partnership (see Table 23), and 
the extent to which SDAs used fixed unit price, performance-based vendor contracts (see Table 
2,6) - and their possible relationship to the disadvantagedness of the population that was 
enrolled in the SDAs (see Tables 72 and 7.3). As in Chapters 5 and 6 (and earlier in this 
chapter), disadvantagedness is defined in terms of three employment barriers (i.e., low 
educational attainment, limited labor market experience, and public assistance receipt) and the 



classroom training in order to have a higher-skilled work force - has also been stated.) Ttble 
7.7 presents information that may help answer this question. It displays the percentage of 
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TABLE 7.7 



PERCENTAGE DISTRIBUTIONS OF SERVICE STRATEGY RECOMMENDATIONS 
FOR ADULT AND YOUTH TREATMENT GROUP MEMBERS, 
BY SDA ORGANIZATIONAL STRUCTURE 



SDA Organizational Structure Xduhi Y^hT 
and Service Strategy Recommendation (%) ^ 

SDAs tha t are gwejmient-Ojgraj^d (a) 

Classroom Training 3g 3 

OJT/JSA 
Other Services 
Total 



39.3 

47.3 31.2 

14.4 29.5 
100.0 loo.'o 



SDAs that are Goverm nent/PIC-Operatcd (V\ 

Classroom Training 4I 3 

OJT/JSA 



Other Services 
Total 



40.5 

2? ' 18.3 
30.1 4i.2 
99.9 100.0 



SDAs that are PJC^Qpera^ fr) 



Classroom Training 20 6 

OJT/JSA 
Other Services 
Total 



28.9 

60.3 49.8 



19-1 21.4 
100.0 100.1 



Difference (d) 



Classroom Training 17 7 

OJT/JSA 
Other Services 



10.4 

13-0 -18.6 



-4.7 8.1 



2^ MDRC c a|culations from Background Momation Fonn tafanMtioii 
collected by MDRC site representatives during the National JTPA Study. 

NOTES: Calculations for this table .re based on data for all treatment group members 
in the 18-month study sample. 

Distributions may not total 100.0 percent because of rounding. 

Tests of statistical significance were not performed. 

(a) This subgroup includes SDAs where the local government entity serves as the JTPA 
fijndmg r cc lpif nt , the JTPA administrative body, and the contractor with local service providers 
nt - s In ™" subgroup are not incorporated and do not have a separate staff. 

(b) This subgroup includes SDAs where the local government entity and the PIC 
share responsibility as the JTPA funding recipient, the JTPA administrative body, and the 
contractor with local service providers. PICs in this subgroup may be incorporated 

and may have a separate staff. 

, mA (G ] This SUbgrou P inc,udes SDAs »e WC serves .s the JTPA funding recipient 

the JTPA administrative body, and the contractor with local service providers. PICs in 
this subgroup are incorporated and have a separate staff. 

(d) This is the recommendation rate difference between government-operated and 
PIC-operated SDAs. 
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number of these barriers treatment group members had The basic questions addressed in this 
discussion concern the extent to which different types of SDAs were likely to have higher 
enrollment rates among treatment group members with two or three of these barriers than 
among treatment group members with none of these barriers. 

1* SPA performance and oomilatioa served Like the view that SDA performance 
may be related to service emphasis, one widely held view in the JTPA system is that higher 
rates of performance among SDAs are associated with an emphasis on serving a less 
disadvantaged population. The reasoning here suggests that, in SDAs that strive to exceed 
performance standards, less employable and more disadvantaged applicants are screen* A out 
Further, it may also be tha' the remaining pool of individuals, being more employable, is better 
suited for OJT or job search assistance, thus accounting for the emphasis on th es e services 
noted above. 

Evidence concerning this hypothesis is presented in Table 7.8, which is similar to Tables 
12 and 73. The table presents the percentage of adults and youths in the employment barrier 
subgroups and their enrollment rates. The SDAs in the study are grouped according to the 
extent to which they exce e d ed their performance standards (using the entered employment rate 
standard for both adults and youths and positive termination rate only for youths). Tests were 
run to determine whether the enrollment rate in sites that substantially crafted their 
standards was statistically different from the enrollment rate in sites that moderately nrerded 
their standards, and from the enrollment rate in sites that marginally rxcrcrird their standards. 
For example, the top panel of Table 7.8 shows the enrollment rate of adults with two or three 
employment barriers in SDAs that substantially exceeded their performance standards (56 
percent) compared with the enrollment rate for this subgroup in SDAs that marginally 
e xceeded their standards (66 percent). The table indicates that this difference was statistically 
significant at the 1 percent leveL 

As in Tables 12 and 73, the last column of Table 1J& shows tb*. enrollment rate 
differences between adults or youths with two or three employment barriers and those with 
none. A positive difference means that the group of SDAs enrolled a greater proportion of 
its more disadvantaged treatment group pool than of its less disadvantaged treatment group 
pooL A negative difference means the reverse. 

This an?lysis, which is subject to the same caveats noted in the discussion of Tables 12 
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and 73, suggests a mixed stoiy for adults and youths. In SDAs that substantially exceeded 
standards, the enrollment rate for adults with two or three employment barriers was 7 
percentage points lower than the enrollment rate for adults with no employment barriers. 
Further, compared with SDAs that marginally exceeded their performance standards, SDAs that 
substantijJly exceeded their standards began with smaller proportions of adults who had 
multiple employment barriers and were less likely to enroll these adults. However, compared 
to SDAs that marginally exceeded their performance standards, SDAs that substantially 
exceeded the standards were also less likely to enroll adults with no employment barriers. 

The opposite story appears to hold for youths when using either entered employment rate 
or positive termination rate as the basis. Here, in SDAs that substantially exceeded their 
entered employment rate standard, the enrollment rate for youths with multiple barriers was 
10 percentage points larger than the enrollment rate for youths with none of the employment 
barriers. The comparable difference for the positive termination rate standard was 14 
percentage points. In each case, these SDAs began with a larger proportion of youths with 
multiple employment barriers than youths with no employment barriers in their pools. 

For adults (but not for youths), SDAs that substantially exceeded their standards tended 
to provide more OJT (and less classroom training), and among all those recommended for a 
service strategy, they tended to enroll a greater proportion of less than of more disadvantaged 
individuals. One possible explanation is that performance standards for youths included 
"positive termination" outcomes such as completion of schooling or attainment of employment 
competencies, not just - as with adults - standards related to job placement. This could have 
permitted, or encouraged, SDAs to enroll a more disadvantaged youth population - for whom 
job placement at termination from JTPA was not seen to be an appropriate goal - without 
jeopardizing their ability to meet performance standards. 

*• SPA organizational structure (nature of the local nartngrship) and pom.lntinn 
served. One hypothesis concerning PIC-operated SDAs is that they would tend to be more 
concerned about meeting employers' needs than the needs of very disadvantaged clients, and 
that they therefore would favor services to more readily employable individuals and those who 
confront fewer barriers to employment This subsection explores this hypothesis using the 
approach employed in the previous subsection. Table 7.9 displays the percentage of adult 
and youth treatment group members in the employment barrier subgroups, along with their 
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enrollment rates. Here the SDAs are grouped according to whether they were predominantly 
PIC-opcrated, predominantly government-operated, or operated by a more balanced 
combination of the PIC and a government entity. For Table 7.9, tests were run to determine 
whether the enrollment rate in PlCoperated SDAs was statistically different from the rate in 
each of the other two groups. For example, the top panel of Table 7.9 shows the enrollment 
rate of adults with two or three employment barriers in PlCoperated SDAs (55 percent) 
compared to the analogous rate in government-operated SDA~ (62 percent). The table 
indicates that this difference was statistically significant at the 5 percent leveL The last column 
in Table 7.9 indicates the difference in enrollment rates between those with two or three 
employment barriers and those with none. A positive difference means that the group of 
SDAs was more likely to enroll those with multiple employment barriers than those with none 
of the barriers. 



The information in Table 7.9 suggests at most a modest and tenuous relationship between 
the nature of the local partnership and the disadvantaging of the population the SDAs 
served. The relationship is also different for adults and youths. For example, in PlCoperated 
SDAs, the enrollment rate for adults with multiple employment barriers was 7 percentage 
points lower than the rate for adults with none of the barriers. In government-operated SDAs 
the enrollment rates for these two subgroups were approximately the same. Interestingly, both 
the PlCoperated and government-operated SDAs began with roughly the same proportion of 
adults with multiple employment barriers. The patterns for youths are different Compared 
to government-operated SDAs, PIC-operated SDAs began with a lower proportion of youths 
with multiple employment barriers, but the enrollment rate for these youths was 4 percentage 
points higher than the rate for those with none of the three barriers. For both adults and 
youths in government-operated SDAs, there was very little difference in enrollment rates for 
the two employment barrier subgroups. 

3 - TypCS of SBdgr ™ ntracts gBAS "se and nonuiotin. ggggd. As described in 
Chapter 2, there was considerable variation in the type of contracts SDAs used for contracted 
services: fixed unit price, performance-based or cost-reimbursement As discussed earlier one 
frequently posed hypothesis is that fixed unit price contracts - since they withhold full payment 
for services until an enrollec is placed on a certain type of job for a particular period of time 
at a specified wage rate - provide strong incentives for contractors to serve a more employable 
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population, in order to be assured of covering their costs. 12 In contrast, the argument states, 
contracts that assure full payment regardless of outcome -"cost-reimbursement" contracts - 
serve more disadvantaged individuals, since there is no monetaiy incentive to work with easier- 
to-place individuals. 

Table 7.10 provides information that may shed some light on this hypothesis. It displays 
the percentage of adult and youth treatment group members in the employment barrier 
subgroups and their enrollment rates. Here the SDAs are grouped according to the percentage 
of total Title II-A training ex inditures that were paid to vendors under fixed unit price, 
performance-based contracts: "high use" if such contract payments exceeded 50 percent of total 
Title H-A training expenditures; "moderate use" if such contract payments were between 15 and 
49 percent of total Title II-A training expenditures; and "little or no use" if such contract 
payments were under 15 percent of total Title H-A training expenditures. In Table 7.10, tests 
were run to determine whether the enrollment rate in SDAs that made high use of fixed unit 
price contracts was statistically different from the enrollment rate in each of the other two 
groups of SDAs. For example, the top panel of Table 7.10 shows the enrollment rate of adults 
with two or three employment barriers in "high use" SDAs (60 percent) compared to the rate 
in "low use" SDAs (61 percent). The table indicates that this difference was not statistically 
significant The last column in Table 7.10 indicates the difference in enrollment rates between 
those with two or three employment barriers and those with none. A positive differences 
means that the group of SDAs was more likely to enroll those with multiple employment 
barriers than those with none of the barriers. 

Table 7.10 does not present any clear evidence supporting this hypothesis for either adults 
or youths. For example, the enrollment rate among adults with multiple employment barriers 
was lower than the rate for adults with none of the barriers in both SDAs that made high use 
of fixed unit price contracts and SDAs that made little or no use of them. It is also notable 
that the high and low users of fixed unit price contracts began with roughly the same 
proportion of adults in both employment barrier subgroups in their pools of persons 
recommended for a service strategy. 



^There are, of course, methods to structure fixed unit price, performance-based contracts to avoid 
this outcome, and instead to promote services to a more disadvantaged population. See National 
Commission for Employment Policy, 1985. However, these methods were not reported to be used in 
the 16 study sites (or to be used widely in the JTPA system). 
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A very different pattern emerged for youths. In all SDAs, the enrollment rate among 
youths with multiple employment barriers was higher than the rate for youths with none. 
However, this enrollment rate difference was much greater in SDAs with high or moderate use 
of fixed unit price contracts (113 and 11.6 percentage points, respectively, compared to 12 
percentage points). In addition, SDAs that made high use of fixed unit price contracts began 
with a higher proportion of youths with multiple employment barriers than did SDAs that made 
little or no use of fixed unit price contracts. The differences between adults and youths may 
be accounted for in part by the ability of SDAs to write youth fixed unit price contracts for 
outcomes other than job placement — the only use to which they could be put for adults. 

In conclusion, the 16 SDAs in the study showed often very large cross-site variations in 
the activities to which they referred applicants, the services enrollees actually received, and the 
enrollment rates of more and less disadvantaged individuals. This information should be useful 
for understanding the findings presented in the 18-month impact report (Bloom et aL, 1993). 

The further analyses and findings in this chapter were subject to a number of very 
important caveats and cautions about the limits of the SDA-level analysis that could be 
undertaken in the National JTPA Study. An examination of the evidence for a number of 
hypotheses revealed few relationships, and in some cases where there was some suggestive 
evidence of a relationship, the findings were notably different for adults and youths. 
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APPENDIX A 



PROFILES OF THE SERVICE DELIVERY AREAS 
IN THE NATIONAL JTPA STUDY 



This appendix contains brief profiles of the 16 service delivery areas (SDAs) that are 
participating in the National JTPA Study. It supplements the material in the body of the 
report Each profile describes the site's location, size, population, and labor market; its 
program (structure and services); and its implementation of the study. The profiles are 
presented in alphabetical order, with the site name followed by the formal SDA name. 

These profiles are intentionally brief and are. meant to provide the reader with some 
understanding of the diversity across sites. Programs not included in the sample, such as Titles 
m and n-B, and services to in-school youths through Title ILA, are not specifically discussed. 

To ensure a consistent base for comparison across sites, 1979 Census data were the 
primary source of information on population and size, while the JTPA Annual Status Report 
(JASR) for 1986 was used for information on labor market conditions. SDAs 9 Annual Job 
Training Plans, and observations and data collected by the researchers during the 
implementation of the study, were also drawn on in developing these profiles. 

The profiles depict the SDAs and their local environments as of the time the random 
assignment process was going on, *jad therefore do not reflect subsequent changes. When 
references are made to the research sample, they refer to all those randomly assigned in the 
site, not just the 18-month study sample. 
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BUTTE. MONT.: CONCENT RATED EMPI/)YMENT PROCttAM 
Study Context 

The mSST^ B F**T!S SS*"* (CEP) KTVCS 10 counties * ^tcrn Montana. 
The CEP SDA has approximately 125,000 residents, of whom about 37,000 live in Butte- 24 000 
live in Heler^ The remaining parts of the SDA are very rural, with no towns over 15,000 in 
population. SDA residents are primarily white, and 7.5 percent of all families had incomes 
below the federal poverty level in 1979. 

the 1 A V°T5^ traditionaU y bascd °n fining, timber, and ranching. In 
the late 1970s and early 1980s, major layoffs in mining and related industries occurred in Butte 
and Anaconda, causing a decline in the local economy in these areas. The state capital in 
Helena provides white-collar employment, while Butte's economy is gradually awersifvine 
Employment in the SDA was 9 percent in 1984 and gradually dedfidTr/SSSS 
LpnSgram ^986° aVCragC annual wa « e for ^kers in the SDA waT $16,700 

The Program 

During the period of this study, the Montana Department of Labor and Industry 
ErtSrS n oh ^. 1 D ™ on ' wa * the Srant recipient for the SDA which has a PIC appointed 
by the CEP Council of County Commissioners^ The Council of County Commissioners also 
is consulted on JTPA policy and at times meets jointly with the PIC The JTPA proeram is 
operated by the Montana Job Service Division, under contract with the Employment Policy 
Dillon. Four Job Service offices provide JTPA services: in Helena and Butte; separate JTPA 
intake offices serve clients, while in Anaconda and Dillon, the Job Service provides both its 
usual job listing services and JTPA in a single office. 

The Job Service staff offer job search assistance and OJT in all offices and provide 
classroon i training by referrals to other agencies. In Helena, a basic skills brush-up course 
followed by clerical training was an important service, preparing people for employment in the 
capual area, while m much of the rest of the SDA OJT was the most common service. In the 
rural parts of the SDA the nearest provider of classroom training in occupational skills was 
many miles away. 

Study Implementation 

The normal intake procedures in the two larger offices (Helena and Butte) differed 
somewhat from those in the smaller offices. In Helena and Butte, intake for JTPA was 
normally done on a group basis, typically scheduled once a week. Clients completed the JTPA 
application ^ and met with staff for a brief assessment interview. An orientation session was 
then held for applicants who were eligible and appropriate for JTPA At this session, staff 
would provide job counseling, conduct a fuller assessment, and then work with individuals to 

k>n July 1, 1990, the operator of this SDA changed from the Montana Department of Labor and 
Industry, a government enuty, to Montana Job Training Partnership, Inc, a private nonprofit 
organization formed by the two PICs in Montana. P non P rom 
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arrange services. In the smaller offices, staff worked with clients individually and the 
procedures could be handled more informally. 

In order to introduce the study procedures in this SDA, three issues had to be addressed. 
First, in the two larger offices, an appropriate point for the designation of the recommended 
service strategy had to be identified that would come late enough to provide staff with 
information on the clients but before services were provided during orientation. Staff agreed 
to base recommendations for services on a review of the materials assembled during the group 
intake sessions. At these sessions, applicants would complete the Background Information 
Form and Informed Consent Form and random assignment was then conducted Only those 
randomly assigned to the treatment group would be scheduled to attend orientation and then 
be referred to sendee providers. 

Second, some clients who were interested in an OJT would try to arrange one on their 
own; this was especially common in the smaller offices. Typically, the percentage of individuals 
who sought and found an OJT in this way was quite low, so random assignment could not 
come before the beginning of this individual OJT search. However, individuals needed to 
understand that if they returned with a possible OJT, enrollment in JTPA was not guaranteed. 
As was always the case, their eligibility would have to be redetermined, and staff would have 
to judge the appropriateness of the job for an OJT, and - during the period of the study - 
they would have to go through random assignment 

Third, in Helena and Butte, agencies serving displaced homemakers provided pre- 
employment skills training and counseling, using state funds, and often then enrolled some of 
their participants into JTPA-funded clerical training. Since the initial portion of the sequence 
was not funded by JTPA and only a portion of participants made it to the JTPA-funded 
services, clients referred from these service providers were excluded from the study. 

The SDA began random assignment in July 1988 and completed it in September 1989, 
falling somewhat short of its original target sample. 

CEDAR RAPIDS. IOWA; EAST CENTRAL IOWA 
Study Context 

The East Central Iowa SDA includes the cities of Cedar Rapids (population 110,000) and 
Iowa City (population 50,000). Total population in the SDA is approximately 330,000, and 
outside the two main cities the SDA is very rural. The largest SDA office is located in Cedar 
Rapids, and each of the surrounding five counties (Benton, Iowa, Johnson, Jones, and 
Washington) has a smaller office. About 6 percent of all families in the SDA had incomes 
below the federal poverty level in 1979. 

The local labor market is closely linked to the fortunes of agriculture, through direct 
production and processing of food products. Within the SDA, the University of Iowa is also 
a major employer. Unemployment remained at about 6 percent from 1984 through 1986, then 
dropped to 4.3 percent in 1987 and to 3.6 percent in 1988 with growth in the local economy. 
The average annual wage in the SDA was approximately $17,200 in program year 1986. 
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The Program 



The SDA (a private, nonprofit, multi-jurisdictional agency) administers the JTPA 
program. Each of its six offices is responsible for taking applications, determining eligibilitv 
and assessing apphcants Staff in the offices are typically experienced, with inanyhaving been 
with the agency since the CETA program. 

Client recruitment was not a problem in this SDA except for occasional problems 
enroUing youths. This was partly because, when the study began, many enroUees were" 
into a new program year. In this second program year, funding was not as great as expected, 
so the target number of new enrollments dropped sharply. «pccieu, 

These funding problems had implications for the services provided in the SDA In oast 
c^a 56 ^ SUCh 85 OJT had ^ major services offered. With the dedte 
in funding, SDA managers encouraged staff to provide less intensive services to allow more 

aTomSA^dl 1 ^ N ^^«^ continued to recommend intensive 
as UJT/JSA and classroom training for most applicants. 

Study Implementation 

Study procedures could be integrated into the normal operations of the SDA with 
relattvdy few change^ In the Cedar Rapids office, apphcatiomfwere taken during a gr^up 
onentation s^ion. During tins session, the staff briefly discussed the study and administered 
short tests of basic skills. Apphcants then met with a coordinator to discuss the program and 

lt ! S!fH any ,^ Cr docu S e f ntation needed to «tablish eligibility. Once ehpbility was 
estabhshed, apphcante returned for an assessment appointment with a coordinator, durmg which 
the Background Information Form and Informed Consent Form were compS On 
completion of assessment, the coordinator recommended individual services and the appropriate 
service strategy for the study and called MDRC for random assignment 

In the five rural offices, application and eligibility determination were done in an 
mdmdual meeting with clients. The Background Information Form and Informed Consent 

SSSSISS? * ^ ^ SSiODS - ^ aSSCSSmcnt P"*** was le« formal than in 

the Cedar Rapids office, and its length varied according to the needs and interests of the client. 
Once assessment was completed, staff recommended a service strategy and called MDRC for 
random assignment 

t >u R ! nd ° m a f^^ t teg** in this SDA in June 1988 and ended in June 1989. Because 
of the sharp decline ui funding for new enrollments during the period of random assignment, 
the s udy sample of 498 fell well below the original target number. The vast majority^ the 

nrT/T e J^V eC ° mm f t l? f ° r , mtenSive * CtivWcs $uch 35 classroom training (55 rx*cint) and 
OJT/JSA (38 percent). General assistance applicants served by the SDA under a state-funded 
welfare employment program were excluded from the study. 



COOSA VALL EY. GA.; COOSA Va jj fry 
Study Context 



The Coosa Valley SDA provides JTPA services for a 10-county area in the northwestern 
part of Georgia. The SDA is large and rural, and public transportation is nonexistent except 
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in Rome, which is the largest city, with a population of 30,000. Services are provided through 
a decentralized structure in order to reach the total SDA population of 355,000, which is 
predominantly (more than 90 percent) white. Approximately 11 percent of the families had 
incomes below the federal poverty level in 1979. 

The most important employment sector is manufacturing, which employed 38 percent of 
the workforce in 1980. The textile and apparel industries dominate this sector. The service 
sector also supplies a significant number of jobs, employing 19 percent of the workforce. The 
unemployment rate started to decline just as random assignment began - from 7 percent in 
1985 to 5.7 percent in 1986. By 1988, however, the unemployment rate had climbed back up 
to 63 percent The average wage in the area was $15300 per year in program year 1986. 

The Program 

A state-created but locally administered Regional Development Center (RDC) is the 
planning and administrative entity and grant recipient for the SDA RDG fulfill multiple 
labor-market-related functions in Georgia and receive funds from the state as well as from the 
programs they oversee, such as JTPA The Coosa Valley RDC provides fiscal and program 
oversight and contracts out all services. The PIC, whose members include a number of the 
JTPA contractors, provides input to the RDC The RDC contracted with Berry College to 
provide intake services as well as occupational training. Because of the large area served by 
the SDA, intake was conducted at provider sites and in central public locations within the 
counties (e.g., courthouses), as well as at Berry College's JTPA office. Following the 
determination of eligibility for services, Berry College staff made referrals to the 18 JTPA 
contractors that dealt with the population covered by the study. 

Community colleges, vocational colleges, technical schools, and private for-profit and not- 
for-profit organizations provided classroom training in a wide range of occupations. 
Community-based organizations worked primarily with out-of-school youth in GED preparation, 
job search activities, and the development of youth competencies. The private contractors also 
arranged OJTs. Contracts for all of the above services were primarily performance-based. 

Study Implementation 

Random assignment started in January 1988 and ended in August 1988. Recruitment was 
not a general problem, although some OJT providers said that the study caused them to lose 
good candidates. Over the study period, there were several organizational changes that 
complicated the implementation process. Berry College had only recently become the 
contractor for intake services and was still refining procedures at the time the study started. 
The good relationship that developed between the service providers, Berry College, and the 
SDA staff was critical to the successful implementation of the study. The need to recommend 
individuals to a service strategy prior to random assignment required further adaptations for 
service providers not providing classroom training in occupational skills. Berry College certified 
and referred applicants to the providers, who were asked to determine whether they would 
accept the applicant. Most contractors for classroom training in occupational skills were already 
doing this type of assessment Once the client was found appropriate for services by the 
contractor, Berry College would complete the random-assignment call and then inform both 
the contractor and client of the client's status in the study. 
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Frequently, applicants interested in classroom training in occupational skills or OJT, 
including some employer-generated referrals, would make their Gist contact with the provider 
rather than with the college* When this occurred, the contractor would complete an assessment 
and then refer the person to the college for an eligibility interview and random assignment 
OJT contractors prior to the study provided several days of job search activities and sometimes 
made referrals to prospective employers prior to the official determination of eligibility. During 
the study, this was restricted to no more than two days of job search and no direct employer 
referrals until after random assignment was completed and the applicant was identified as in 
the treatment group. 

The intake procedures for a contractor who arranged customized training to fill employer 
needs sometimes varied slightly from the procedures described above. In most cases, intake 
personnel went on-site and, if the applicant appeared to be JTPA-eligible, random assignment 
was conducted prior to obtaining all documentation to support the eligibility. This was done 
to expedite the placement of individuals in these programs. Those who were assigned to the 
treatment group were then required to complete the certification process; any who did not 
meet all eligibility requirements were not enrolled by the SDA 

Approximately 40 percent of all applicants were recommended for classroom training over 
the random-assignment phase of the study, with 34 percent of the applicants recommended for 
OJT/JSA Special groups excluded from the study in this SDA were individuals served with 
3 percent funds or 6 percent funds, which were targeted for hard-to serve individuals (a 15- 
slot program for people with multiple barriers to employment), people with handicaps, who 
were to be served by a vocational rehabilitation contractor, individuals in a special program for 
the deaf, and people served through a limited-slot reciprocal agreement with the neighboring 
SDA. 

Adjustments to performance standards were not required in either year of the study. 



CORPUS CH RISTI. TEX; CORPUS CHRISTI/NUECES COUNTY 
Study Context 

The Corpus Christi/Nueces County SDA (previously called the Corpus Christi Job 
Training Program) is located in southeastern Texas on the Gulf of Mexico. Corpus Christi, 
with a population of 232,000, has 86 percent of the 270,000 county residents. Hispanics and 
whites have been represented almost equally in the population, at 47 percent each, but the 
Hispanic population is increasing. Black residents represent about 5 percent Approximately 
13 percent of the families in the SDA had incomes below the federal poverty level in 1979. 

The local economy, with many jobs tied to the oil industry as well as tourism, was 
affected by the major recession in Texas in the mid-1980s. The unemployment rate fluctuated 
from 9.1 percent in 1984 to 12 percent in 1987 to 10.4 percent in 1988. The average annual 
wage for the area in program year 1986 was $18,500. 

The Program 

At the beginning of program year 1988, the PIC was incorporated and assumed 
responsibility as grant recipient and administrative entity. Staff then became employees of the 
PIC. There are two JTPA offices in the SDA, although the site outside the city is not a full- 

170 




service center. The SDA is responsible for Title II-A programs, including 3 percent and 8 
percent programs, as well as Titles II-B and ID. 

OJT and classroom training in occupational skills are the most frequently emphasized 
services, with some GED and job search services also provided Except for assessment and 
intake, the PIC uses performance-based contracts and subcontracts all sendees to outside 
organizations. The primary classroom training contractor, a community college, provides 
classroom training in a wide range of occupational areas; JTPA participants are mainstreamed 
with other students. A GED class, however, is offered only to JTPA clients. The Texas 
Employment Commission, the public Employment Service, is currently responsible for 
administering the OJT program, although in the first year of the study, a private for-profit 
contractor fulfilled this function. Two community-based organizations are also major 
contractors to the SDA: LULAC conducts job search, job club, and job placement, originally 
as a stand-alone activity, but more recently as a follow-up component for those who have 
completed classroom training in occupational skills at the community college; SER provides 
concurrent GED and classroom skills training for out-of-school youths. The SDA expanded 
its own services to include some job search activities. Cost-reimbursement contracts were used 
for individual referrals to specific vocational training courses, offered primarily through 
proprietary schools. 

Study Implementation 

Random assignment began in January 1988 and ended in January 1989, with 1,609 people 
randomly assigned Generally, recruitment was not a problem, although specific contractors 
reported some difficulty meeting goals; the SDA felt this was only partly caused by the 
establishment of the control group. During the first year of random assignment, all JTPA- 
funded students at the community college were excluded from the sample because enrollment 
in classes had already occurred. In the second year, the SDA redefined procedures and only 
new students (or students who had taken only preparatory courses) who were randomly 
assigned to the treatment group were allowed to receive JTPA funding. Excluded from the 
sample were older individuals served through 3 percent funding, the 8 percent program, a small 
number of AEDC recipients in a GED program, and a program funded through Title II-A 6 
percent funds that covered work and training-related expenses, such as uniforms and car 
repairs, for AFDC recipients. 

Implementation of the study did require closer coordination between the SDA and its 
contractors, although relationships prior to the study were already good. Responsibility for 
recruitment was shared: the SDA conducted assessments and determined eligibility, and the 
contractors were responsible for additional assessments and random assignment Applicants 
who were recruited and seen first by the contractor were sent to the SDA for eligibility 
determination and referred back to the contractor for random assignment. Applicants who 
were recruited by the SDA staff were assessed and screened by the service providers prior to 
random assignment. Those determined inappropriate by the service providers were referred 
back to the SDA for further assessment. There were some problems with the flow of people 
and paper as a result of this process. Originally, paperwork was given to the applicant to take 
with him or her to the other agency, but some of it was getting lost Procedures were modified 
so that the contractor picked up a copy of the paperwork and could track who was being 
referred. 
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During the sample build-up period, 50 percent of the applicants were recommended for 
classroom training and 42 percent were called in for OJT/JSA The random-assignment process 
varied slightly by provider. To expedite employer referrals, the OJT contractor completed the 
call while the client waited. The contractor who focused on out-of-school youths scheduled 
applicants in groups, completed the call-in for random assignment at the end of an orientation 
session, and divided the group to inform them of the results. The community college made the 
random-assignment call for all applicants just prior to the start of a new semester. 

The SDA did experience some problems with performance over the period of the study. 
During the first year, funds were slightly underspent and the governor anproved an adjustment 
In the second year, the SDA requested an adjustment in the pc*t-program follow-uo 
employment rate for adults. K 

DECATUR. ITJ- ? MArnM/r >E WITT COUNTIES 
Study Context 

The Macon/De Witt Counties SDA, located in central Illinois east of Springfield, serves 
the two counties. They have a total population of nearly 150,000. Decatur, a city of 
approximately 90,000 located in Macon County, is the home of the larger JTPA office- a 
second office is located in De Witt County in the town of Clinton, which has about 8,000 
residents. Outside of Decatur, the SDA is primarily rural, with scattered small towns. Decatur 
has a significant black population, but the remaining parts of the SDA are primarily white. In 
1979, 7.8 percent of the families in the SDA had incomes below the federal poverty level. 

The Macon/De Witt labor market continued to experience high unemployment 
throughout the 1980s. The unemployment rate was 10.7 percent in 1984, 102 percent in 1985 
and 113 percent in 1986; it declined slightly to 10.6 percent in 1987 and to 93 percent in 
1988. Plant closings or layofis in several large heavy manufacturing firms occurred in the 
1980s, and these cuts caused a leveling off in wholesale trade, finance-insurance, and real estate 
employment Construction of a nuclear power plant in Clinton did increase the demand for 
labor in that part of the SDA The average annual wage in the SDA was $22300 in nrocram 
year 1986. v 6 

The Program 

The JTPA grant recipient is Macon County, which administers the program. An 
experienced staff operate the program, with the director and assistant director having served 
in the agency since the CETA program. The two JTPA offices in Decatur and Clinton conduct 
their own intake, assessment and service delivery. In light of the high unemployment rate in 
the SDA staff have not needed to actively recruit clients; 40 percent of applicants have been 
walk-ins, and the bulk of the remainder have been referrals from public aid or the Employment 
Service. 

The primary services offered by the SDA are OJT and job search assistance, though 
classroom training in occupational skills and basic education were planned to increase in size 
during the period of the study. At the time the study started, a local community college 
operated the OJT and job search assistance activities. These were provided under an 
arrangement by which the SDA paid a specified portion of the salaries of college staff, with 
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the payment unrelated to the number of people served, though the college received bonuses 
when it placed certain groups in jobs. The SDA also funds basic education (through a regional 
school district) and work experience, with a combination of these two activities sometimes 
serving as a lead into an OJT. 

Study Implementation 

The study procedures could be introduced in this SDA with relatively few changes in 
normal operations, partly because the existing high demand for the program meant that demand 
for program services clearly exceeded available program slots. When clients initially inquired 
about the program, staff provided them with information on program services and eligibility 
rules and scheduled an individual intake interview with a counselor. At that appointment, the 
counselor explained the study, provided an additional orientation to the program, filled out an 
application and Background Information Form with the client, and began the assessment 
process. Once a week, the counselors met with the supervisor to agree on an appropriate 
service plan for applicants. Random-assignment calls were made following these meetings. 
Seventy-seven percent of those randomly assigned were recommended for the OJT/JSA service 
strategy. 

Only one special change was made in the usual procedures to implement the study. Prior 
to the start of random assignment, SDA staff had referred applicants recommended for either 
OJT or job search assistance to the local community college providing these services, without 
designating which service should be provided Prior to the study, SDA staff were considering 
designating a single service at referral and used the start of random assignment (with its 
requirement of a single recommended service strategy) to introduce this change. The only 
group specially excluded from the study in this SDA was older workers recommended for a 
special OJT program. 

Random assignment began in November 1987 and continued through July 1988, with the 
total sample reaching 471, below the original target number. The SDA chose to end random 
assignment early in order to participate in a state-initiated economic development project, which 
required them to refer large numbers of carefully screened job candidates for possible 
employment at a new plant in the area. 



FORT WAYNE, IND.: NORTHEAST INDIANA 
Study Context 

The Fort Wayne Area Job Training and Development Corporation or JobWorks, located 
in Indiana's northeastern corner, serves eight counties: Adams, Allen, De Kalb, Huntington, 
Noble, Steuben, Wells, and Whitley. (Northeast Indiana is the SDA name.) Fort Wayne, a 
city of approximately 175,000 in Allen County, is the largest urban center in the SDA and the 
site of the SDA's central office. Outside of Allen County, the SDA is primarily rural, with 
many small towns. The SDA's total population is approximately 500,000, about 93 percent of 
whom are white, with black residents making up the bulk of the remaining population. An 
estimated 6 percent of the families in the Fort Wayne area had incomes below the federal 
poverty level in 1979. 
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A strong recovery from the recession of the carry 1980s dropped the unemployment rate 
from 8.9 percent in 1984, to 5.1 percent in 1987, to 4.8 percent in 1988. Some of the rural 
counties had unemployment rates well below this level During 1988 and 1989, employers were 
having difficulties filling jobs, and some were seeking workers from outside the area. 
Manufacturing remains the largest employer in the SDA, providing jobs for one-third of all 
workers in 1987. Between 1983 and mid-1987, manufacturing employment increased by 26 
percent or 16,500 jobs, though many of the new manufacturing jobs paid less than heavy 
industry jobs lost in the late 1970s and early 1980s. The average wage in the Fort Wavne 
area was $17,800 in program year 1986. 

The Program 

t i.™, 7 ? 6 PIC * an mcor P°rated nonprofit organization, which has an agreement with 
u en? to administcr toe JTPA program through an office in each of the eight counties in 
the SDA Each JobWorks office is responsible for its own intake, assessment, service delivery 
and - during the period of the study - random assignment A state imtiative to merge the 
activities of the Indiana Employment Security Division (the state Employment Service) and 
JTPA led to greater cooperation between these two agencies at the local level and co-location 
of offices in many of the rural counties surrounding Fort Wayne. Even before this most 
JobWorks applicants had already registered with the Employment Security Division, and those 
who had not were required to register as part of JTPA intake. 

JobWorks offers a wide range of services, with the program emphasis varying by local 
office depending on the availability of service providers. The primary services offered are OJT 
job search assistance, and - in the areas close to Fort Wayne - classroom training in' 
occupational skills. OJT and job search assistance are operated by the JobWorks staff, while 
all classroom training in occupational skills programs are run by service providers, most of 
whom operate under performance-based contracts. The site subcontracts with about 20 service 
providers, with the usual number of clients ranging from 5 to 40 per program. Service 
providers include local technical institutes and schools, community-based organizations, and 
other organizations that offer occupational training in a wide variety of areas or work with 
specific populations such as youths or handicapped individuals. 

Study Implementation 

The intake process varied slightly from office to office, but generally random assignment 
occurred after the client had completed all application forms, been determined eligible for 
JTPA been assessed by JobWorks staff, and been recommended for a specific service strategy 
The Background Information Form for the study was completed along with other application 
materials, and the study was explained as part of the initial intake session. The Informed 
Consent Form was signed during an assessment interview. In the Fort Wayne office 
assessment often involved extensive testing. In order to accommodate the study design 
JobWorks staff in the Fort Wayne office agreed to reduce job search assistance during 
assessment to approximately two hours. 

Service strategy recommendations were made at the weekly staff meetings in the larger 
offices and on an individual basis in the smaller offices. Fifty-eight percent of all persons 
randomly assigned were recommended for OJT/ISA 33 percent for other services, and 10 
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percent for classroom training. Random-assignment calls were made on a batched basis at 
scheduled times, though individual calls at other times were also possible. 

Over the course of random assignment, client recruitment became more of a problem 
because of the decline in the unemployment rate. The SDA made extensive changes in 
recruitment materials and introduced new procedures to retain a higher percentage of 
applicants in the program. Two small programs (one funded by the state for welfare recipients 
as a lead into JTPA and a second for hard-to-serve youths) experienced special recruitment 
problems and were exempt from the study. Staff did notice that over time they were working 
with less skilled clients. 

Random assignment began in this SDA in November 1987 and continued through August 
1989, reaching a total sample of 3,608. 



HEARTLAND. FLA.: HEARTLAND 
Study Context 

The Heartland SDA serves five counties (De Soto, Hardee, Highlands, Polk, and 
Okeechobee) in central Florida between Tampa and Orlando. Seventy-five percent of the 
SDA's population resides in Polk County, the site of Lakeland (population 50,000), the largest 
city in the SDA Winterhaven, a retirement community, is also in Polk County. The remaining 
four counties are much less developed, though they are dotted with a number of cities and 
towns. Eleven percent of families living in the SDA had incomes below the federal poverty 
level in 1979. 

Employment in the SDA is a mixture of manufacturing, services, and (near Orlando) 
tourism. The unemployment rate in the SDA stood at 113 percent in program year 1984, 
declining gradually to 9.4 percent in 1987 and to 8.1 percent in 1988. During the early 1980s, 
a major aircraft manufacturing facility closed, but after the start of the study, retail trade 
employment rose sharply with the opening of a major regional shopping center. Average 
annual wages in program year 1986 were $15,200, among the lowest of the SDAs in the study. 

The Program 

The Heartland Private Industry Council (PIC), a nonprofit organization, operates the 
JTPA program within the SDA At the time the study began, clients were served through 
seven offices, three of which were in Polk County. OJTs are arranged by PIC staff, bui 
classroom training in occupational skills is subcontracted to approximately 10 service providers 
in a typical year, with the Polk County Board of Education being the largest Though these 
service providers do recruit clients, the PIC staff conduct the JTPA intake and eligibility 
determination. The Florida Employment Security Commission (ESC) offices in Lakeland and 
Winterhaven also provide job search assistance under a contract with the PIC and do conduct 
JTPA intake. 

Classroom training in occupational skills is the predominant service provided by the SDA 
In program year 1986, two-thirds of all adult participants and almost 40 percent of youths 
received this service. While recruitment had not been a problem in the mid-1980s, finding 
appropriate JTPA-eligible applicants for the more demanding courses could be difficult. At the 
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time the study began, the SDA planned to put somewhat more emphasis on OJT in its service 



plan. 

Study Implementation 



Since 80 percent of JTPA clients come from Polk County, implementation of the studv 
w« limited to that area. The Florida ESC offices in the county were also parTof the s S 

Intake and eligibility determination were done in individual meetings with clients during 
which the study was explained and the Background Information Form aSCmS S 
Form were completed. Following this, eligible applicants were referred to ^menf^an 
extensive array of testing and counseling lasting up to six hours and culmmating in the" 
development of an employment development plan. Staff then designated a recommend^ 
service strategy for the study and reviewed with L applicant once agaiftle Jorm^n^ 
SEL^Lf"? T?T nt T COm P lctcd ' staff «■* MDRC for random assignment and 
?? ° f *" S P ecW <ari « lons fr° m Ae study included a liimCnumber 

of extreme hardship cases and applicants who had completed a state-funded remedia^Sfon 
program and returned to JTPA for further training. remcuiai education 

500 J^nf^^Sf rand | om ^ i * nment in May 1988. By October 1988, when only about 
500 persons had been randomly assigned, staff were concerned about the dramatic drop in 
applicants and the shortfall in enrollments. This was occurring because of a Se fa the 
unemployment rate in the SDA, the opening of the major retail center, and p oblems^it £ 
clients because of publicity about funding cuts in the SDA for JTPA services. 

SJESS^T^r* 6 3 DCW ChaUengC requirin * morc fatensive recruitmt 

S?„* AT*. a Pf uca ° te . and ncw procedures to address the needs of hard-to-serve 

23L J^ P 1 lCChniCal ° n dicDt ""^ and rete ^°> the SDA 

continued to experience a major enrollment shortfall. After initially suspending random 
assignment for two months, the SDA decided to end it in January 1989. 

JACKSON. MISS.: CAPITAL /IPFi 
Study Context 

t u cities "» northcentral Mississippi are served by the Capital Area SDA Thev are 

Jackson Clinton, and Pearl, with Jackson having 85 percent of the total population of 239 000 

2223 TFV* ?5 SDA ' S W"™™ * WaCk and ° ther minorit ^ A» cstLatedTs 
percent of the families had incomes below the federal poverty level in 1979 

to T^l^U 3 *, 6 9 P 6 "** m 1985 and 6 7 I*™* in 1986. and declined 

TJZT 1* T * ^ largCSt of j Qbs 15 the Sector. The state cSln 

Jackson provides white-collar employment Other service industries, such as hospitals are 

JSmSsT" $OUrCC ° f emp,0yment PaC avcra « e wa « e in the SE> A v/as $17,200 in program 
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The Program 



The SDA is a consortium, with the city of Jackson serving as the administrative and 
planning entity and grant recipient for the program. Staff, employed by the city in the 
Department of Human and Cultural Services, are housed in one central office. The SDA is 
the grant recipient for Title II-A and II-B funds, but does not operate Title II-A 3 and 8 
percent programs or Title HI. 

The SDA offers classroom training in a variety of occupational areas. Performance- 
based contracts are used to fund training in specific vocational areas such as health occupations, 
auto mechanics, truck driving, cooking, and security, JTPA is the primary source of business 
for a number of these contractors. Other contractors, including a community college and 
Jackson State University, are funded to provide other services, including a 6- to 8-week job 
club/employability development program, which can lead to an OJT, and ABE and GED 
classes, A total of 12 contractors were utilized during the study period. In addition, SDA staff 
develop OJT positions and place participants in employment Some of the OJTs are developed 
for employer-referred candidates. 

Study Implementation 

Service providers are responsible for their own recruitment The SDA has experienced 
some difficulty reaching enrollment targets for the white population. Sixty percent of the 
eligibility interviews are conducted by city JTPA staff at the SDA office, and 40 percent are 
held at contractor sites prior to the start of new classes. Some applicants are screened by the 
provider and then sent to the SDA office for the eligibility interview. When eligibility could 
be determined without additional follow-up during the initial interview, the random-assignment 
phone call took place immediately. Otherwise, random assignment occurred after additional 
documentation had been obtained, without the applicant being present. After random 
assignment, those assigned to the treatment group were referred to the service providers, who 
determined whom they would enroll. 

A variety of release and information forms, in addition to those required for the study, 
are required by the state for those enrolled in the program. Several months after random 
assignment began, procedures were modified so that these additional forms were completed 
only for individuals assigned to the treatment group. 

The start-up of the study in February 1988 coincided with the release of additional funds 
to the SDA by the state. Therefore, the SDA had to issue additional Requests for Proposals 
and quickly initiate new programs just as assignments to the control group were beginning to 
occur. The timing of these two events created some problems for the SDA, and a performance 
adjustment was eventually needed over two program years, with random assignment ending in 
March 1989. Nevertheless, the SDA met its random-assignment sample goal several months 
early and extended random assignment for one month in order to increase the sample above 
the original goal. 

During the period of the study, 41 percent of the applicants were recommended for 
classroom training, 37 percent for other services, and 22 percent for OJT/JSA. Individuals 
served through a contract with vocational rehabilitation services were excluded from random 
assignment. 
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JERSEY CITY. N J.: CORPORATION FOR EMPLOYMENT AND TRAINING, INC. 



Study Contort 

The Corporation for Employment and Training (referred to in this report as Jersey City) 
provides JTPA services to a single city area* This urban area, with a population of 224,000, 
is approximately 50 percent black, with the remainder of the residents divided primarily among 
whites, Hispanics, and Middle Eastern and Asian immigrants. With a population density of 
7,000 per square mile and 147 percent of the families with incomes below the federal poverty 
level in 1979, the area shares problems similar to those confronting other large urban areas. 

The overall average wage during the year preceding implementation of the study was 
$20,400. This average masks great differences in income within the community, with many 
longtime minority residents earning much less and many recent arrivals in "gentrified" 
neighborhoods working in high-paid jobs elsewhere in the metropolitan area. The 
unemployment rate dropped from 10.9 percent in 1984 to 7.9 percent in 1987 and then to 12 
percent in 1988. However, many residents have sought employment outside the SDA because 
of a severe decline in manufacturing and the closing of many factories. The service sector, 
including financial institutions and distribution centers, are major employers, but wage rates are 
lower than in neighboring New York City. 

The Program 

The city contracted with the Corporation for Employment and Training, a private, non- 
profit organization that previously operated the CETA program, to administer the JTPA 
program. 2 The corporation administered the Title II-A 3 percent and 8 percent programs and 
Title HI in addition to the 78 percent and 6 percent funds, which are passed through to SDAs 
by the state. 

The OJT program was implemented by in-house staff, who also provided direct 
placements and job search. The corporation subcontracted for classroom training in 
occupational skills, using a total of 23 providers throughout the study period. Contracts were 
performance-based and often directed to specific service occupations. Training was provided 
in areas such as clerical, financial, health and food services, and drafting occupations. One 
provider received funding to provide placement services to the Hispanic population during part 
of the study. Proprietary and business schools as well as vocational technical centers were 
primary contractors. The SDA increased its emphasis on individual referrals and contracts 
toward the end of the study implementation period. 

Study Implementation 

Applicants' first contact with the program was through the corporation or the contractor. 
Eligibility was determined at the corporation's intake office, and some brief testing was 
conducted at that time. Applicants who were assessed and determined eligible were called in 
for random assignment at the end of the day. People assigned to the treatment group were 



2 The Corporation for Employment and Training lost its contract with Jersey City at the end of 
June 1990. 
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notified of their status by phone or mail Toward the end of random assignment, procedures 
were modified and random assignment was completed while the applicant waited so that those 
assigned for sendees could be immediately referred to an appropriate service provider and 
enrolled by the end of the program year. 

The corporation reported some difficulty recruiting youth throughout the study period, 
and some providers reported special recruitment problems. The random-assignment ratio was 
adjusted at several point and then briefly suspended in older for the SDA to tiy to meet goals 
for the program yean / lthough the SDA met all the performance standards during the first 
year of the study, its performance was slightly down from the previous year, particularly in the 
entered employment rate for adults and youth. A performance adjustment was requested from 
the state, since this decline in performance affected the levels of incentive funds the SDA 
received A small adjustment was approved. 

Random assignment started in November 1987 and ended in September 1989, with a total 
of 1,686 people in the sample. About 40 percent of the sample were recommended for 
classroom training, while the other 60 percent were divided almost equally between OJT/JSA 
and the "other services" strategy. The 3 percent program was excluded from the sample, as 
v.-ere homeless people who had to be in a job training program to qualify for shelter. 



LARIMER COUNTY. COLO,: LARIMER COUNTY 
Study Context 

Located in the northcentral part of the state, approximately 65 miles north of Denver, 
this rural county had a total population of 150,000 in 1980, 65 percent of whom lived in Fort 
Collins or Loveland, site of the two JTPA offices. By the mid-1980s, the SDA's population 
was approximately 200,000 The population is predominantly white (92 percent in 1980); the 
largest minority is Hispanic (6 percent). Only 5.9 percent of the families were classified as 
having incomes below the federal poverty level in 1979. 

The county experienced strong growth in the late 1970s and early 1980s, but in the years 
immediately preceding the study, unemployment began to rise — from 4.8 percent in 1984 to 
7.4 percent in 1987. In 1988, however, unemployment had receded to 6.6 percent. The service 
sector employs the largest number of workers, followed by trade and manufacturing. Colorado 
State University is the largest employer, with Hewlett Packard and Anheuser-Busch next in 
number of employees. The overall average wage in Larimer County was $17,500 in program 
year 1986. 

The Program 

The county government is the planning and administrative entity and grant recipient for 
JTPA SDA staff are county employees in a department called Larimer County Employment 
and Training Services (ETS), but an active PIC guides the program. The SDA also administers 
the Title II-A 3 percent program and Title III. Fort Collins is the administrative headquarters 
and serves about 60 percent of the SDA's clients, with the Loveland office serving the 
remainder. When the study started, the SDA and PIC also oversaw the local Employment 
Service activities and staff, *vhich are co-located with the SDA offices. But actions at the state 
level gradually resulted in a return to the more traditional, separate administrative structure. 
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ETS, because of its relatively small Hue H-A 78 percent allocation, serves the majority 
of its clients through short-term, low-cost, but highly structured activities, many conducted by 
its own personnel: job club, a three-week program with its own training area and phone bank, 
is offered monthly in each office; a one-week assessment program and youth competency 
workshops are also ongoing. A workshop to teach job retention skills was added. 

Other programs are contracted to schools on a cost-reimbursement basis. Individuals 
recommended for the classroom training service strategy begin their time in training with a two- 
week career planning program for JTPA participants conducted at Front Range Community 
College. ETS staff contribute to curriculum development and consult with instructors and 
dients in these programs. The high level of interaction between staff and other agencies is 
also evidenced by a formal network of 19 community organizations, including groups not 
mvolyed with JTPA who meet regularly to foster coordination and develop solutions for 
mdmdual problem cases. The SDA has received national recognition for its networking system 

ETS, through the PIC, has also established itself as an employer resource in two key 
ways. First, several major companies have used ETS and the Employment Service as a center 
for screening and testing prospective employees. While both JTPA-eligible people and others 
who are not disadvantaged have been placed through this service, few of the large employers 
use the OJT program. Second, as a service for employers, the PIC annually sponsors a 
Business Expo, bringing in experts to educate area employers. 

Study Implementation 

Before the study was implemented in December 1987, enrollment occurred prior to 
assessment in order to discourage the development of a screening process that might eliminate 
the harder-to-scrve. To accommodate the research design, the SDA decided to implement a 
mini-assessment (locally called a pre-EDP) immediately following eligibility determination so 
that applicants could be recommended for a service strategy prior to random assignment ETS 
worked with a consultant to develop a checklist that was used to assist staff in determining an 
applicant's appropriateness for each of the three service strategies. 

Following the pre-EDP, the random-assignment call was placed and staff informed people 
of then- research status in person. People assigned to the treatment group received additional 
assessment either on the same day or shortly thereafter. Because of the large number of 
participants served in job search and assessment, the "other services" strategy for this SDA was 
set at the comparatively high level of 70 percent During random assignment, 64 percent of 
the applicants were recommended for this strategy, and an unusually high percentage ended 
up enrolling only in miscellaneous services (sec Tables 73 and 7.4). Twenty percent of the 
people randomly assigned were recommended for classroom training in occupational skills, with 
only 7 percent of those randomly assigned recommended for the OJT/JSA service strategy 

To avoid jeopardizing ETS's good relationship with other agencies, ETS and the 
researchers conducted briefings on the study procedures for community agencies prior to the 
start-up of random assignment Groups not included in the research in this SDA were all 
individuals 55 and over and referrals for service from Project Self-Sufficiency (PSS). PSS is 
a federally funded demonstration project designed to increase the self-sufficiency of AFDC 
recipients. It uses the resources of multiple agencies to provide comprehensive services - 
including housing, employment and training services - to help people make the transition from 
welfare to work. 
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Midway through random assignment, for about six months, the SDA also conducted group 
orientations prior to the eligibility interview. During these motivational sessions, a localized 
version of the random assignment videotape was shown. Recruitment activities did not change 
substantially during the sample build-up period, and particularly in the Loveland office 
enrollment goals were difficult to meet 

Random assignment ended in Larimer County on September 30, 1989, with 1,027 people 
in the sample. 



MARION. OHIO: CRAV^ORD/HANCQCK/MARIONAVYANDOT COUNTIES 
Study Contort 

Located in northcentral Ohio, this four-county SDA has a population of 206,000. Marion 
is the largest city, followed by Findlay; each has a population of about 36,000. The population 
is predominantly white. Approximately 7 percent of the families had incomes below the federal 
poverty level in 1979. 

Economically, there is considerable variation. The county of Hancock has had the lowest 
unemployment rate in Ohio, while the other counties have been adversely affected by declines 
in manufacturing. Overall, the unemployment rate for the area, which averaged 10.2 percent 
between 1984 and 1986, dropped to 8.3 percent in 1987 and to 6.8 percent in 1988. The 
average annual wage in program year 1986 was $18,100. 

The Program 

The PIC is incorporated and serves as the administrative entity and grant recipient. Its 
staff includes employment advisors, testers, and basic education instructors. The central 
administrative office is in Marion. Five JTPA training centers are spread throughout the SDA 
and are staffed by PIC personnel, who conduct extensive assessments and provide case 
management services. Through the training centers, participants can receive basic education 
and GED courses, using computer-assisted instruction. PIC staff also conduct job club and 
arrange work experience and OJT placements. Classroom training in occupational skills is 
arranged on an individual basis through reimbursement contracts with vocational schools and 
technical colleges. 

Participants formerly averaged approximately 40 weeks in the program as they moved 
through a variety of components designed to upgrade their skills. However, the decrease in 
the unemployment rate, relative to other SDAs in the state, resulted in a reduction in Title II- 
A funds. 

Study Implementation 

Random assignment began in January 1988 and ended in July 1989, with 1,154 people 
in the sample, of whom more than 1,000 were recommended for services in either classroom 
training (49 percent) or OJT/JSA (40 percent). Exempted from the study were older 
individuals served with 3 percent funds and job-ready welfare recipients mandated to participate 
in a job club under a contract with the Department of Human Services in two of the counties 
(at one point, a third county also participated in this project). 
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Because the Title H-A funding was reduced, and fewer people could be served, there 
were no recruitment problems during the period of random assignment In the spring of 1989, 
enrollments for all groups except AEDC recipients were frozen because of the fundine 
shortage. * 

The need to identify service strategies prior to random assignment caused the SDA to 
lengthen its assessment process in some cases to ensure that applicants were assigned to the 
correct services. Assessments were individualized, but all applicants were given the same 
battery of tests during half-day assessment sessions. The testing technician also provided the 
general orientation to the program and briefly explained the study. After the tests and 
determination of eligibility, applicants met with an employment advisor to make their goals 
final. For some people, determination of the appropriate service strategy was a short process- 
for others, it could extend over several days or weeks. Employment advisors would then discuss* 
their cases at a staff meeting so that the final decision could be made about an appropriate 
service strategy. Subsequently, all cases handled in that week were called in for random 
assignment Applicants were notified of their status by phone or letter, dependine on how 
soon a service was scheduled to begin. 



NORTHWES T MINNESOTA: NORTHWEST MINNESOTA 
Study Context 

The Northwest Minnesota Private Industry Council (PIC) serves seven counties (Kittson 
Marshall, Norman, Pennington, Polk, Red Lake, and Roseau), which comprise a very rural area 
in the comer of the state abutting North Dakota and Canada. The population of the entire 
SDA is about 100,000, and its three largest towns (Crookston, East Grand Forks, and Thief 
River Falls) each has slightly under 10,000 residents. The population is primarily white, and 
about 11 percent of all families had incomes under the federal poverty level in 1979. 

The local economy is agriculture-based, with farrning, food processing, and related trade 
being the major industries. The severe winters make for a short growing season, and sugar 
beets, potatoes, and wheat are the primary crops. The unemployment rate in the SDA stood 
at 103 percent in 1984 and remained above 9 percent until 1987, when it dropped to 8.6 
percent By 1988 unemployment had fallen to 8 percent The average annual wage of $13,600 
in program year 1986 was the lowest of any SDA in the study. 

The Program 

The PIC is an incorporated nonprofit organization that is the JTPA grant recipient. It 
does some client recruitment but the high unemployment rate has allowed the JTPA program 
to rely primarily on walk-ins and referrals from other social service and education agencies 
The Minnesota Job Service operates most of the JTPA program in the SDA under a 
subcontract from the PIC In offices in Crookston and Thief River Falls, Job Service staff 
offer the usual state Employment Service job listings plus JTPA and other state-funded 
programs for welfare recipients. The local community action agency conducts intake for a small 
older workers' program, which provides work experience and job search assistance. 
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Job Service staff directly provide job search assistance, career exploration, OJT, and work 
experience. They refer people interested in classroom training in occupational skills to the 
local community college, technical institute, or campus of the state university. Those seeking 
adult basic education and a GED are referred to a local adult learning center for services. A 
local technical institute also provides an intensive job search assistance course, primarily for 
those participating in classroom training in occupational skills. OJT and classroom training in 
occupational skills are the two most common Title II- A services. 

Study Implementation 

Job Service staff offered all those who inquired about their services an orientation to the 
various programs available* Those who appeared to be eligible for JTPA and interested in its 
services then completed a JTPA application* Job Service staff then determined JTPA eligibility, 
conducted an assessment of needs, and designated appropriate services. 

Study procedures were integrated into the usual Job Service intake by completion of the 
Background Information Form and Informed Consent Form during an initial meeting with the 
client In Thief River Falls, much of this paperwork was done in a group, while the Crookston 
staff completed study forms during an individual meeting with clients. Choice of the 
appropriate service strategy was typically made based on an individual meeting with the client, 
during which an assessment of needs, interests, and current skills was made. Staff then called 
MDRC for random assignment and informed the client of the result and next steps. 

The main issue that had to be addressed in implementing the study concerned a state- 
funded welfare employment program (PATHS). The Job Service had a contractual obligation 
to use JTPA funds to serve welfare recipients referred by the county welfare departments. 
These referrals were excluded from random assignment and were not a part of the study. 

Random assignment began in August 1988 and ended in May 1989. The SDA slightly 
exceeded its target sample of 550. Virtually all those randomly assigned had been 
recommended for intensive services such as OJT/JSA (69 percent) and classroom training (31 
percent). 



OAKLAND. CALIF.: OAKLAND 
Study Context 

The City of Oakland, with a population of 340,000, is one of 49 service delivery areas 
in California. Its population is approximately 40 percent white, 30 percent black, 25 percent 
Asian, and 5 percent other minorities. There are 6,300 residents per square mile, and 8.7 
percent of the families had incomes below the federal poverty level in 1979. 

Major industries are transportation (including shipping), manufacturing, government, and 
health care. Large employers are Kaiser, Pacific Bell, Merritt-Peralta Medical Supplies, Civic 
Gas and Electric, and Childrens Hospital. The unemployment rate fluctuated in the years 
preceding the study's implementation: it was 7.0 percent in 1984, 6.8 percent in 1985, 8.7 
percent in 1986, 7.7 percent in 1987, and 6.6 percent in 1988. The overall average annual 
wage in the city was $22,000 in program year 1986. 
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The Program 



The PIC is incorporated and shares administrative responsibility for the program with the 
City of Oakland. The city is also the grant recipient and planning entity for the program. The 
PIC and city have defined their oversight and administrative functions so that the citv is 
^£"5^. detennining applicants' eligibility for programs and for tracking participation 
m the SDA's programs, while the PIC awards and monitors contracts for services. 

A subsidiary of the PIC - the Oakland Private Sector Corporation (OPSC) - provides 
direct client services. The SDA also contracts with numerous outside agencies to provide 
services to the eligible population. Outside of OPSC, contracts tend to be targeted to serve 
particular subgroups or to provide training through one of three recommended sendee strategies 
defined for the research. All subcontracts are performance-based 

Classroom training in occupational skills is the predominant service. Seven of the 
contractors (community-based, private for-profit, and nonprofit organizations) provided 
c assroom training in occupational skills. Two of these also had contracts to provide OJT 
along with another contractor who was also funded to develop OJTs, primarily with the Korean' 
population. Two other contractors provided job search services, with one of them targeting 
services to people 55 and over and the other enrolling veterans. 

OPSC, the largest contractor, provides classroom training in occupational skills OJT and 
other services to aU JTPA-eligibles and conducts an older workers* program with Title H.-A 3 
percent and 78 percent funds. At the same time the study began, a new program, funded 
through Title II-A 6 percent funds, was initiated to provide training in nontraditional jobs. The 
Center for Employment and Training (CET), the second largest contractor, conducts training 
programs in four occupational areas - office occupations, shipping and receiving, auto 
mechanics, and maintenance - and provides a small number of OJTs. 

Study Implementation 

The procedures for the study were integrated into the normal operations of the SDA 
with relatively few changes in the flow. However, the involvement of multiple organizations 
providing different services to different subgroups, made the process of adapting the study to 
the site complex. Because of multiple population subgroups with varying degrees of proficiency 
in English, the agreement to participate form was translated into nine languages. In addition 
the exclusion of several programs and all youths from the study sample complicated the initial' 
screening process. v 

Youths were excluded from the study because of recruitment problems with this group 
A residential treatment program for court-referred offenders was excluded. Also excluded was 
a basic education program funded through Tide II-A 8 percent funds. These funds were 
earned over from the previous year and then targeted to serve AFDC recipients in a state 
welfare-to-work initiative. Also exempted from the study were a program provided through an 
agreement with the Department of Vocational Rehabilitation to place people with handicaps 
m jobs and a program for homeless individuals (paid for with 6 percent funds) 

i ^?° Ugh ^ ac ' ore ^ reSpODsible for raniitment for their own programs, people 
also applied to JTPA by going to the SDA's central intake unit In order to identic the 
appropriate service strategy for applicants, providers were divided into two groups: those that 
offered only one major category of service (classroom training in occupational skills, OJT or 
job search) and those that could offer multiple types of services. The first group was labeled 
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"single service providers" and the second group, represented by GET and OPSC, "multiple 
service providers." The following description of the flow is the same for both groups, except 
that GET and OPSC completed most of their assessment, recommended people for service 
strategies, and placed the random-assignment phone call themselves after eligibility had been 
determined 

In most cases, clients applied for programs directly at the service provider. In these 
instances, contractors conducted program orientations and assessments, which varied in length 
and intensity. During the initial meeting(s), the study was explained and the Background 
Information Form (BIF) was given to the applicant along with other intake documents needed 
for the eligibility interview. Contractors, particularly those that served predominantly limited- 
English-speaking clients, sometimes helped applicants complete the BIE Eligibility interviews 
with city staff were then scheduled at either the central intake unit or the service provider site. 
The BIF was finalized, the agreement to participate form signed, and the random-assignment 
phone call made during this interview, assuming all supporting documentation was available. 

As noted above, some clients applied directly to the JTPA office. Applicants whose 
initial point of contact was the central intake unit had their eligibility determined and the study 
explained. Then they signed the BIF and agreement to participate form. After a brief 
assessment of interest, the appropriate activity category was identified and random assignment 
took place, followed by the referral of those in the treatment group to the appropriate 
provider. 

Random assignment began in July 1988 and ended in March 1989, with 1,072 people 
randomly assigned, slightly more than the targeted sample. Classroom training was the 
predominant recommended service strategy, with 50 percent of the sample recommended for 
this strategy; 8 percent were recommended for OJT/JSA. 



OMAHA, NEB.: JOB TRAINING OF GREATER OMAHA 
Study Context 

Located in eastern Nebraska, this SDA serves approximately 500,000 people and includes 
the greater Omaha metropolitan area. The largest county is Douglas, with a population of 
398,000, of whom 312,000 live within the Omaha city limits. The SDA also includes Sarpy 
(population 86,000) and Washington (population 16,000) counties. The majority of the 
population is white, with black residents making up the largest minority group. In 1979, 6.6 
percent of all families had incomes below the federal poverty level. 

The employment base includes large manufacturing and transportation employers as well 
as jobs in clerical and administrative work and sales and sales-related employment. The 
Strategic Air Command (S.A.C.) is the largest employer, followed by other large service-sector 
firms. 

The unemployment rate ranged between 4.7 percent and 5.6 percent in the four years 
preceding the study. In 1987, it was 5 percent, dropping to 4.4 percent in 1988. The average 
annual wage was $17,700 in program year 1986. 
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The Program 



The Job Training of Greater Omaha SDA (formerly called the Omaha Office of 
Employment Resources) provides JTPA services in the three-county area. The city of Omaha 

^ h ^, adnU S atlVe ™ Ut * ^ d grant red P ient » but 40 active PIC guides the program and is 
closely involved in major decisions. The SDA also administers Tide IH Eight pelicntftmds 
are used to enroU people in basic education classes combined with work amSeSS Reseat 
usuaUy foUowed by OJT or classroom training in occupational skills. EignTp^^n fi*b Z 
ato used for classroom training in occupational skills and other classes af JS I coWun^ 

Classroom training in occupational skills is the predominant service for Tide H-A JTPA 
participants. During implementation of the study, most participants irTSS SS^tSi 
£u£ maj ° rCOntraCt0rS ' predominantly communi^aJo^ 
traimng in service occupations, such as health services, cooking, finance, officTLuWrn 

E^^,^ .Contracts were performance-based. Job search b enfa2*£ 
house staff, who also monitor and arrange OJT placements. ^ 

Study Implementation 

tn t^I h I SerViCe u?r OVide l rS r L CSp ° nsiblc for recruitment, a function that had been assigned 
to them the year before the study began. However, the SDA also provided general recntiS 
for the program as a whole. Intake, consisting of completion of an appucation JS 
determination was normally performed at the central Omaha office. (The office^ moved to a 
new locanon in early 1989.) During the intake interview, the Background Info^toTponn 
w^ fflled out by staff and the study was briefly explained. Eligible applicants v£re Z 
referred to an assessment and testmg workshop, also conducted by SDA staff At the 
workshop the videotaped explanation of the study was shown and the Informed Consent Form 

Z/!?™ mCCU f S™ 1 * COm P lctin S thc assessment and testing workshop were 

used to approve counselors' recommendations of services. 

People recommended for the classroom training service strategy were referred to the 
appropriate contractor for a provider's assessment People determined appropriate by die 

SE2? TV m hCdUlCd t0tan cm P lo y*™ty development plan (EDP) intervieWwith 
SDA staff which could take place in-pcrson or by phone. After the EDP interview, SDA 
staff completed random assignment and notified the contractor and applicant of his or her 
status. People recommended for OJT/JSA were scheduled to meet with Z SDA job developer 
for an EDP interview immediately following the staff meetings discussed above Random 
assignment took place after the plan was made final. Kandom 
_ . J° mplanem the study, several changes were made in the assessment and tcstinc 
workshop. Previously, the workshop had extended over two days and was followed for some 
apphcants, by a two-day job search assistance workshop. However, the SDA had difficulty 
retaining people through this whole process and requested on-site training by Cygnet AsS« 
prior to *e start of the study. As a result of the training, the workshop IXZ to^ 
half-day. The SDA also sought to discourage contractors from conducting an additional 
assessment, although some continued to require their own job-readiness assessment 

Seventy-three percent of the people randomly assigned were recommended for the 
classroom training service strategy. OJT/JSA was the second most frequently recommended 
service strategy, accounting for about 22 percent of the sample. 
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Random assignment began in October 1988 and continued through September 1989, with 
1362 people randomly assigned In program year 1988, the SDA did not meet the adult 
entered employment rate standard, and the director did not request an adjustment In program 
year 1969, the SDA revived adjustments in its adult and welfare entered employment rate 
standards. Hie original sample goal of 1,600 was not met because two months before the study 
ended, the SDA was authorized to exempt from random assignment applicants recruited and 
referred for intake by the contractors. 



PROVIDENCE, R.L: PROVIDENCE/CRANSTON 
Study Context 

The Providence/Cranston SDA serves these two adjacent, older urban areas in northeast 
Rhode Island. Providence, with a population of approximately 155,000, is the state capital and 
largest city in the state, while Cranston has approximately 70,000 residents. About 8 percent 
of the residents of the SDA are blacks, and all minority groups comprise 13 percent of the 
total population. Nine percent of families in the SDA had incomes below the poverty level 
in 1979. 

The Providence/Cranston area has historically been one of the most industrialized in the 
country, with a higher than usual percentage of employment in manufacturing. In recent years, 
several of the state's major employers (makers of jewelry and silverware and the U.S. Navy) 
cut back employment, but other sectors (including services) grew with the New England 
recovery of the early and mid-1980s. Unemployment in the SDA stood at 7.2 percent in 1984, 
but dropped to 43 percent in 1987 and to 3.7 percent in 1988 as the SDA beneflted from the 
then-booming regional high-tech and services economy. The average annual wage in the SDA 
in program year 1986 was $17,000. 

The Program 

The Providence/Cranston Job Training Partnership (PCJTP) is the grant recipient and 
administers the program through offices in each of the two cities in the SDA Recruitment 
of clients, done by both the PCJTP and service providers, has become increasingly a challenge 
as the area's unemployment rate dropped throughout the mid-1980s. Intake, eligibility 
determination) assessment, and counseling are performed by the SDA staff in each office, 
though the Cranston office must file case documentation of eligibility in the main Providence 
office before assessment can be scheduled. 

PCJTP staff arrange OJTs for clients, but the remainder of SDA services are provided 
by subcontractors operating under a mix of performance-based and cost-reimbursement 
contracts. Among the services provided in this way have been adult basic education, English 
as a second language, classroom training in occupational skills, vocational exploration programs 
(pre-employment skills and work experience for youths), and job clubs. 

Study Implementation 

PCJTP staff conduct intake, eligibility determination, and assessment of applicants. The 
Background Information Form and Informed Consent Form were completed as part of an 
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assessment interview. During the assessment interview, the counselor determined whether 
further testing was required. If not, as was often the case for those recommended for OJT, 
job search assistance, and vocational exploration, staff called MDRC for random assignment 
during the assessment interview and informed the client of the result at that time. If further 
testing was needed, as was possible for classroom training in occupational skills, this additional 
assessment was completed before random assignment occurred. 

The increasing difficulty the SDA faced in recruiting clients because of the decline in the 
area's unemployment rate created issues in the implementation of the study. Early in random 
assignment, the SDA encountered serious problems recruiting youths and meeting the required 
40 percent of expenditures for services to youths. This was soon followed by problems 
recruiting adults. To allow continuation of the study, the random-assignment ratio was changed 
to ease the recruitment difficulties. Service providers also did less intensive testing of 
applicants, and some shifted to individualized open/entry, open/exit services as opposed to 
traditional group training. 

The SDA did not meet two of its adult standards in program year 1987, in part because 
of the attention paid throughout the year to the problems in youth programs. The state did 
not adopt adjustments in the performance standards suggested by the U.S. Department of 
Labor, and incentive funds to die SDA declined as a result 

Random assignment began in this SDA in November 1987 and ended in September 1989, 
with the SDA meeting its sample target 



SPRINGFIELD. MO.; TOB COUNCIL OF THE OZARKS 
Study Context 

The Job Council of the Ozarks SDA located in southwestern Missouri, serves seven 
counties: Christian, Dallas, Greene, Polk, Stone, Taney, and Webster. Springfield (population 
133,000), in Greene County, is the largest city and the location of the SDA's central office. 
The SDA is primarily rural, with a total population of 304,000, of whom more than 90 percent 
are white. An estimated 10 percent of the families had incomes below the federal poverty level 
in 1979. 

The economy has been strong, with employment increasing in both the service sector, 
which employs the majority of the labor force, and manufacturing. Greene County also has 
had a strong agribusiness base. The unemployment rate has gradually declined in recent years, 
from 7.7 percent in 1984, to 5.7 percent in 1987, to 53 percent in 1988; in areas that are 
strongly influenced by tourism, seasonal changes can bring the rate as high as 20 percent The 
average wage in program year 1986 was $15,200. 

The Program 

The City of Springfield Human Resources Department is the administrative entity and 
grant recipient JTPA services are provided through the Job Council of the Ozarks, with full- 
service offices in Springfield, Branson, and Bolivar, a regularly staffed outreach office in 
Buffalo, and application sites in Kimberling City and Marshfield. Staff in each office are 
responsible for intake, assessment service delivery, and implementation of the study procedures. 
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The PIC and local elected officials act as policymakers and planners and provide program 
oversight 

Most of the classroom training in occupational skills programs are located in the 
Springfield area and transportation is a problem for many SDA residents, so there is heavy 
reliance on OJT programs to meet the population's training needs. Classroom training in 
occupational skills was provided primarily in health occupations, with programs such as licensed 
practical nursing and respiratory therapy* Welding, office occupations, and auto mechanics were 
also among the offerings. GED training was provided to dropout youths through a 
reimbursement contract with a community-based organization. Youths also received pre- 
employment skills training and job placement services. Work experience and a job-seekers' 
clinic were provided for both adults and youths. 

The Job Council of the Ozarks consistently had high outcomes on performance standards. 
However, in program year 1986, the average wage for adults employed fell slightly below the 
model-adjusted standard. 

Study Implementation 

The SDA made significant changes in its recruitment, intake, and assessment process 
midway through the implementation of random assignment Using the training provided by 
Cygnet Associates, it streamlined its enrollment process, eliminating an initial screening 
interview and some tests and emphasizing program benefits. SDA staff developed a variety of 
materials to explain and facilitate implementation of the study. 

In the Springfield office, applicants were scheduled for motivational group orientations, 
dui!ng which the study was explained, the Informed Consent Form signed, and the Background 
Information Form (BIF) handed oat with a list of other documents the applicant would need 
for the eligibility interview. Applicants were then scheduled to see a technician, who 
determined eligibility, briefly assessed the applicant to determine appropriate activities, and 
completed the BIF. While the applicant waited, a clerk made the random-assignment call and 
the applicant was informed of the results. People assigned to the treatment group were then 
referred to the next step in the service plan, which was agreed to during the assessment. 

Procedures for the other, smaller offices did not include the group orientation. 
Applicants were informed in-person or by phone about the study and the paperwork needed 
to determine eligibility. Whenever feasible, the eligibility interview and assessment and random- 
assignment phone call occurred during a single visit 

Recruitment was difficult owing to the low unemployment rate, and staff reported that 
they did relax some of their entiy requirements during the study. Because of various concerns, 
a decision was made to shorten the lenfth of random assignment from 17 to 12 months. The 
study did affect the SDA's ability to fully expend Title H-A funds, and the SDA said thai 
performance declined, although performance standards were met 

Over the study period, about 65 percent of those randomly assigned were recommended 
for the OJT/JSA service strategy, about 17 percent were recommended for the classroom 
training strategy, and 18 percent were recommended for the other service strategy. The 3 
percent program, administered outside the SDA, was excluded from the study, as was the 
licensed practical nurse training. The research design was modified to enable the SDA to add 
a short OJT component for some classroom training participants. 

Random assignment began in April 1988 and continued through March 1989. A total 
of 1,202 people were assigned for the study sample. 
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APPENDIX B 

SUPPLEMENTAL TABLES TO CHAPTERS 4. 5. AND 6 



TABLE B.l 



PERCENTAGE DISTRIBUTIONS OF SERVICE STRATEGY RECOMMENDATIONS 
FOR ADULT MALE TREATMENT GROUP MEMBERS, 
BY SELECTED BASELINE CHARACTERISTICS 



Characteristic 
and Subgroup 



Sample 
Size 



Classroom 
Training 

(*) 



OJT/JSA 
(%) 



Other 
Services 
(%) 



All Adult Male Treatment 
Group Members 

Age 
22-29 
30-44 
45 and over 

Ethnicity 
White, non-Hispanic 
Black, non-Hispanic 
Hispanic 
Other 

Education 
High school diploma or GED 
No high school diploma or GED 

Work Experience 
Worked 13 weeks or more 

in 12 months before assignment 
Worked fewer than 13 weeks 
in 12 months before assignment 

Public Assistance Receipt 
Not receiving AFDC or other cash 

assistance at assignment 
Receiving AFDC or other cash 

assistance at assignment 

Barriers to Employment (a) 
None 
1 
2 
3 



3,759 



1,676 
1,628 
455 



2,142 
1,084 
365 
168 



2,410 
1,092 



1,977 
1,405 



24.6 



26.6 
23.6 ** 
21.1 ** 



19.6 

33.7 *** 
21.9 

35.7 *** 



27.0 

20.1 *** 



25.7 
24.0 



48.7 



48.1 
49.6 
48.1 



59.6 

29.8 *** 

48.8 *** 

32.1 *** 



48.3 
51.1 



26.4 



25.4 
26.8 
30.8 ** 



20.8 

36.5 *** 

29.3 *** 

32.1 *** 



24.7 

28.8 *** 



50.8 23.4 
47.6 * 28.4 *** 



2,770 


24.5 


48.6 


26.8 


546 


27.5 


55.5 *** 


17.0 


1,503 


26.7 


48.0 


25.3 


1,508 


22.9 ** 


49.1 


28.0 


607 


24.7 


49.8 


25.5 


107 


18.7 * 


57.9 ** 


23.4 



(continued) 
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TABLE B.l (continued) 



Classroom Other 

Characteristic Sample Training OJT/JSA Services 

and Subgroup Size : (%) (%) (%) 

AFDC History 

Never received AFDC 3,419 24.3 48.7 27.0 
Received AFDC less than 

2 years (b) 233 28.3 48.5 23.2 
Received AFDC 2 years 

or more (b) 58 20.7 69.0 *** 10.3 *** 

Household Composition 

Spouse present 1 ,270 23 .9 53 .9 22. 1 

No spouse present, child present 281 24.6 42.7 *** 32.7 *** 
No spouse present, no child 

present 1,885 25.1 48.1 *** 26.8 *** 



SOURCE: MDRC calculations from Background Information Form responses. 

NOTES: Calculations for this table are based on data for all adult male treatment 
group members in the 18-month study sample. The total sample size may vary among 
characteristics because of missing data for some treatment group members. 

Distributions may not total 100.0 percent because of rounding. 

For each characteristic, within each service strategy, a chi-square test 
was applied to the difference in service strategy recommendation rates between the 
first subgroup listed and, separately, each of the remaining subgroups. Statistical 
significance levels are indicated as * * 10 percent; ** « 5 percent; *** = 1 percent. 

(a) The barriers to employment include: not having a high school diploma 
or GED at the time of random assignment; having worked fewer than 13 weeks 
during the 12 months prior to random assignment; and receiving AFDC or other 
cash assistance at the time of random assignment. 

(b) The AFDC receipt may not have been continuous. 
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TABLE B.2 



PERCENTAGE DISTRIBUTIONS OF SERVICE STRATEGY RECOMMENDATIONS 
FOR ADULT FEMALE TREATMENT GROUP MEMBERS, 
BY SELECTED BASELINE CHARACTERISTICS 



Characteristic 
and Subgroup 


Sample 
Size 


Classroom 
Training 

(«) 


OJT/JSA 
(«) 


v/iucr 
Services 
(%) 


All Adult Female Treatment 












Group Members 




aa n 
44.U 


35.0 




21.0 


Age 












22-29 


1,974 


46.8 


34.3 




18.8 


30-44 


1,922 


45.6 


33.6 




20.8 


45 and over 


569 


29.2 *** 


41.8 


*** 


29.0 *** 


Ethnicity 












White, non-Hispanic 


2,433 


39.8 


40.2 




20.1 


Black, non-Hispanic 


1,377 


48.3 


31.2 




20.5 


Hispanic 


507 


50.5 *** 


23.5 


*** 


26.0 *** 


v/iner 


148 


52.0 *** 


24.3 


*** 


23.6 


Education 












High school diploma or GED 


2 98Q 




35.3 




19.4 


No high school diploma or GED 


1,174 


Af\ Q *** 


35.9 




O AAA 

23.2 *** 


Work Experience 












Worked 13 weeks or more 












in 12 months before assignment 


1,797 


41.4 


38.3 




20.3 


*TUJ1»U ACWCI I llM II ID WCCJlS 












lfl 1 TtirmtHc V^fXr a ibmiwhwii* 

uj ii, uHJuuia uciurc ■ssignmeni 




47.9 *** 


31.7 


*** 


20.3 


Public Assistance Receipt 












Not receiving AFDC or other cash 












assistance at assignment 


A, t HJ / 


1*7 A 


39.8 




22.8 


Receiving AFDC or other cash 












assistance at assignment 


1,683 


54.7 *** 


27.5 


*** 


17.8 *** 


Barriers to Employment (a) 












None 


1,314 


37.7 


40.5 




21.8 


1 


1,624 


42.2 ** 


36.4 


»» 


21.4 


2 


1,160 


50.8 *** 


29.9 


*** 


19.3 


3 


335 


53.7 *** 


26.3 


*** 


20.0 
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TABLE B.2 (continued) 



Characteristic 
and Subgroup 



Classroom Other 
Sample Training OJT/JSA Services 
Size (%) (%) (%) 



AFDC Histoiy 

Never received AFDC 2, 174 
Received AFDC less than 

2 years (b) 1,000 
Received AFDC 2 years 

or more (b) 1,210 

Household Composition 
Spouse present 920 
No spouse present, child present 2,088 
No spouse present, no child 
present 855 



37.3 



39.7 



23.0 



47.4 *** 33.0 *** 19.6 ** 
53.1 *** 28.7 *** 13.3 *** 



41.2 39.1 19.7 

50.5 *** 31.7 *** 17.8 



30.9 *** 41.1 



28.1 *** 



SOURCE: MDRC calculations from Background Information Form responses. 

NOTES: Calculations for this table are based on data for all adult female treatment 
group members in the 18-month study sample. The total sample size may vary among 
characteristics because of missing data for some treatment group members. 

Distributions may not total 100.0 percent because of rounding. 

For each characteristic, within each service strategy, a chi-square test 
was applied to the difference in service strategy recommendation rates between the 
first subgroup listed and, separately, each of the remaining subgroups. Statistical 
significance levels are indicated as * « 10 percent; ** ~ 5 percent; *** « 1 percent. 

(a) The barriers to employment include: not having a high school diplo ma 
or GED at the time of random assignment; having worked fewer than 13 weeks 
during the 12 months prior to random assignment; and receiving AFDC or other 
cash assistance at the time of random assignment. 

(b) The AFDC receipt may not have been continuous. 




TABLE B.3 



PERCENTAGE DISTRIBUTIONS OF SERVICE STRATEGY RECOMMENDATIONS 
FOR MALE YOUTH TREATMENT GROUP MEMBERS, 
BY SELECTED BASELINE CHARACTERISTICS 



Characteristic 

•nH Qiihorruin 
tUU OULNtlUlip 


Sample 

Size 


Classroom 
Training 
(%) 


OJT/JSA 
(*) 


Other 
Services 
(%) 


au ivuuc i uuui x reaunent 














Group Members 


1,436 


29.9 


32.9 




37.3 




Age 














16-19 


892 


33.2 


27.4 








20-21 


544 


24.4 *** 


At 0 


*** 


33. o 




Ethnicity 














White, non-Hispanic 


771 


27.2 


42.4 




30.4 




Black, non-Hispanic 


424 


25.5 


19.1 


*** 


55.4 


*** 


Hispanic 


211 


50.2 *** 


27.5 


*** 


22.3 


** 


utner 


30 


16.7 


20.0 


** 


63.3 


*** 


Education 














Hiffh school dtnlnm* nr OT*T^ 


jM 


LI Ji 


47.0 




25.2 




No hiffh school riinl rani t\r OPT) 




5X.2. 


22.9 


*** 


45.9 


*** 


Work Experience 














Worked 13 weeks or more 














in 12 months before assignment 


694 


28.2 


41.4 




30.4 




wor&cu iewer man 13 weeks 














m 12 months before assignment 


616 


32.1 


25.6 


*** 


42.2 


*** 


Pubh Assistance Receipt 














N> receiving AFDC or other cash 














assistance at assignment 


1,120 


29.1 


34.4 




36.5 




Receiving AFDC or other cash 














assistance at assignment 


158 


35.4 


31.0 




33.5 




Barriers to Employment (a) 














None 


371 


25.6 


48.8 




25.6 




1 


607 


30.1 


33.1 


*** 


36.7 


*** 


2 


384 


33.3 ** 


19.5 


*** 


47.1 


*** 


3 


71 


32.4 


19.7 


*+» 


47.9 


*** 



(continued) 
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TABLE B.3 (continued) 



Characteristic 
and Subgroup 



Classroom Other 
Sample Training OJT/JSA Services 
Size (%) (%) (%) 



AFDC History 
Never received AFDC 1,406 
Received AFDC less than 

2 years (b) 20 
Received AFDC 2 years 

or more (b) 3 

Household Composition 
Spouse present 147 
No spouse present, child present 58 
No spouse present, no child 
present 1,159 



29.9 



35.0 



32.6 



60.0 ** 



37.6 



5.0 ** 



0.0 (c) 0.0 (c) 100.0 (c) 



21.1 54.4 24.5 

39.7 *** 27.6 *** 32.8 



30.5 ** 



31.3 *** 38.2 *** 



SOURCE: MDRC calculations from Background Information Form responses. 

NOTES: Calculations for this table are based on data for all male youth treatment 
group members in the 18-month study sample. The total sample size may vary among 
characteristics because of missing data for some treatment group members. 

Distributions may not total 100.0 percent because of rounding. 

For each characteristic, within each service strategy, a chi-square test 
was applied to the difference in service strategy recommendation rates between the 
first subgroup listed and, separately, each of the remaining subgroups. Statistical 
significance levels are indicated as * = 10 percent; ** = 5 percent; *** = 1 percent. 

(a) The barriers to employment include: not having a high school diploma 
or GED at the time of random assignment; having worked fewer than 13 weeks 
during the 12 months prior to random assignment; and receiving AFDC or other 
cash assistance at the time of random assignment. 

(b) The AFDC receipt may not have been continuous. 

(c) A chi-square test was inappropriate because of low expected cell frequencies. 
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TABLE B.4 



PERCENTAGE DISTRIBUTIONS OF SERVICE STRATEGY RECOMMENDATIONS 
FOR FEMALE YOUTH TREATMENT GROUP MEMBERS, 
BY SELECTED BASELINE CHARACTERISTICS 



Characteristic 
and Subgroup 



Sample 
Size 



Classroom 
Training 



OJT/JSA 



Other 
Services 



All Female Youth Treatment 
Group Members 

Age 

16-19 
20-21 

Ethnicity 
White, non-Hispanic 
Black, non-Hispanic 
Hispanic 
Other 

Education 
High school diploma or GED 
No high school diploma or GED 

Work Experience 
Worked 13 weeks or more 

in 12 months before assignment 
Worked fewer than 13 weeks 

in 12 months before assignment 

Public Assistance Receipt 
Not receiving AFDC or other cash 

assistance at assignment 
Receiving AFDC or other cash 
assistance at assignment 

Barriers to Employment (a) 
None 
1 
2 
3 



1,814 


44.3 


23.2 




32.5 




1,077 


43.2 


21.0 




35.8 




737 


45.9 


26.5 


*** 


27.7 


*** 


908 


45.2 


27.8 




27.1 




S89 


33.8 *** 


18.0 


*** 


48.2 


*** 


287 


62.7 *** 


19.2 


*** 


18.1 


*** 


30 


46.7 


26.7 




26.7 




877 


45.3 


29.0 




25.8 




848 


43.5 


16.5 


*** 


40.0 


*** 


643 


AS, 1 


3U.9 




24.0 




nn 


45.0 


17.5 


*** 


37.5 


*** 


,126 


41.7 


26.6 




31.6 




548 


50.4 ♦** 


17.7 


*** 


31.9 




415 


41.2 


35.9 




22.9 




634 


43.8 


25.2 


*** 


30.9 


*** 


535 


46.2 


15.7 


*** 


38.1 


*** 


225 


46.7 


12.0 


*** 


41.3 


*** 
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TABLE B.4 (continued) 



Classroom Other 

Characteristic Sample Training OJT/JSA Services 

and Subgroup Size (%) (%) (%) 

AFDC History 

Never received AFDC 1,287 42.3 24.7 32.9 
Received AFDC less than 

2 years (b) 384 47.9 * 20.3 * 31.8 
Received AFDC 2 years 

or more (b) 129 52.7 ** 16.3 ** 31.0 

Household Composition 

Spouse present 197 45.7 28.9 25.4 

No spouse present, child present 607 46.1 21.9 ** 32.0 * 
No spouse present, no child 

present 880 42.0 23.6 34.3 ** 



SOURCE: MDRC calculations from Background Information Form responses. 

NOTES: Calculations for this table are based on data for all female youth treatment 
group members in the 18-month study sample. The total sample size may vary among 
characteristics because of missing data for some treatment group members. 

Distributions may not total 100.0 percent because of rounding. 

For each characteristic, within each service strategy, a chi-square test 
was applied to the difference in service strategy recommendation rates between the 
first subgroup listed and, separately, each of the remaining subgroups. Statistical 
significance levels are indicated as * * 10 percent; ** *= 5 percent; *** = 1 percent. 

(a) The barriers to employment include: not having a high school diploma 
or GED at the time of random assignment; having worked fewer than 13 weeks 
during the 12 months prior to random assignment; and receiving AFDC or other 
cash assistance at the time of random assignment 

(b) The AFDC receipt may not have been continuous. 
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TABLE B.5 



ENROLLMENT RATES FOR ADULT MALE TREATMENT GROUP MEMBERS 
BY SELECTED BASELINE CHARACTERISTICS AND SERVICE STRATEGY ' 



Characteristic 
and Subgroup 


Ail 

Strategies 

(%) 


Classroom 
Training 
(%) 


OJT/JSA 
(%) 


Other 
Services 
(%) 


All Adult Male Treatment 
Group Members 


60.8 


71.2 


56.6 


58.9 


Age 
22-29 
30-44 
45 and over 


62.0 
59.5 
61.1 


71.5 
69.8 
76.0 


60.4 

52.7 *** 
57.1 


55.1 

63.2 ** 
57.1 


Ethnicity 
White, non-Hispanic 
Black, non-Hispanic 
Hispanic 
Other 


61.6 
57.8 ** 
64.9 
61.3 


71.9 
70.4 
72.5 
70.0 


59.4 

47.7 *** 

53.4 

53.7 


58.0 
54.5 

78.5 *** 
59.3 


Education 
nign scnooj diploma or GED 
No high school diploma or GED 


62.3 
58.3 ** 


71.7 
68.5 


57.7 
55.4 


60.8 
56.5 


Work Experience 
Worked 13 weeks or more 

in 12 months before assignment 
worKcu iewer tnan 13 weeks 
in 12 months before assignment 


62.0 
60.2 


70.7 
72.1 


58.7 
52.9 ** 


59.6 
62.4 


Public Assistance Receipt 
Not receiving AFDC or other cash 

assistance at assignment 
Receiving AFDC or other cash 
assistance at assignment 


58.8 


/1.2 
67.3 


57.3 
54.8 


59.5 
58.1 


Barriers to Employment (a) 
None 
1 
2 
3 


63.1 
60.1 * 
58.6 * 
57.9 


71.6 
73.4 
66.0 
70.0 


60.6 

53.8 *** 
54.0 *• 
56.5 


59.1 
60.2 
60.6 
52.0 



(continued) 
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TABLE B.5 (continued) 



All Classroom Other 

Characteristic Strategies Training OJT/JSA Services 

and Subgroup (%) (%) (%) (%) 

AFDC History 

Never received AFDC 60.5 71.3 56.2 58.5 
Received AFDC less than 

2 years (b) 68.2 ** 74.2 61.9 74.1 ** 
Received AFDC 2 years 

or more (b) 63.8 58.3 62.5 83.3 

Household Composition 

Spouse present 63.0 71.7 59.9 61.2 

No spouse present, child present 59.4 66.7 62.5 50.0 * 
No spouse present, no child 

Present 59.5 ** 71.5 52.7 *** 60.4 



SOURCE: MDRC calculations from Background Information Form responses and 
program enrollment data from the 16 SDAs. 

NOTES: Calculations for this table are based on data for all adult male treatment 
group members in the 18-month study sample. The total sample size may vary among 
characteristics because of missing data for some treatment group members. 

For each characteristic, within each service strategy, a chi-square test 
was applied to the difference in enrollment rates between the first subgroup 
listed and, separately, each of the remaining subgroups. Statistical significance 
levels are indicated as * ~ 10 percent; ** *= 5 percent; *** « 1 percent. 

(a) The barriers to employment include: not having a high school diploma 
or GED at the time of random assignment; having worked fewer than 13 weeks 
during the 12 months prior to random assignment; and receiving AFDC or other 
cash assistance at the time of random assignment. 

(b) The AFDC receipt may not have been continuous. 
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TABLE B.6 



ENROLLMENT RATES FOR ADULT FEMALE TREATMENT GROUP MEMBERS 
BY SELECTED BASELINE CHARACTERISTICS AND SERVICE STRATEGY ' 



Characteristic 
and Subgroup 



All Classroom Other 
Strategies Training OJT/JSA Services 
(*) (*) (%) (%^ 



All Adult Female Treatment 
Group Members 

Age 
22-29 
30-44 
45 and over 

Ethnicity 
White, non-Hispanic 
Black, non-Hispanic 
Hispanic 
Other 

Education 
High school diploma or GED 
No high school diploma or GED 

Work Experience 
Worked 13 weeks or more 

in 12 months before assignment 
Worked fewer than 13 weeks 
in 12 months before ass ignme nt 

Public Assistance Receipt 
Not receiving AFDC or other cash 

assistance at assignment 
Receiving AFDC or other cash 

assistance at assignment 

Barriers to Employment (a) 
None 
1 
2 
3 



64.6 


72.8 


55.4 


62.4 


64.1 


71.3 


54.6 


63.7 


66.1 


74.7 


55.1 


65.0 


61.0 


, 71.7 


58.8 


53.3 ** 


66.7 


74.8 


58.1 


67.6 



58.5 *** 66.3 *** 52.8 * 48.6 *** 



69.4 
70.3 



79.7 
81.8 



42.9 *** 73.5 



55.6 



60.0 



66-6 74.2 57.4 65.4 

60.1 *** 69.2 ** 52.4 * 55.9 *** 



66.9 


75.4 




58.3 


65.8 


63.7 ** 


71.5 


* 


52.7 ** 


62.4 


65.2 


75.7 




56.6 


63.1 


64.4 


70.4 


** 


54.2 


61.9 


66.9 


77.4 




58.5 


64.5 


64.4 


72.3 


*» 


55.5 


64.0 


63.4 * 


71.6 


** 


52.2 * 


59.4 


60.0 ** 


66.1 


**» 


50.0 


56.7 
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TABLE B.6 (continued) 





All 


Clififiroom 

VMHHM WU1 




Cither 


Characteristic 

VAWl WwA 




Tritnmo 


\JJ 1 /J o/\ 


Services 


and Subgroup 


\70) 


{70) 


{70) 


{70) 


AFDC History 










Never received AFDC 


64.1 


73J 


55.9 


62.8 


Received AFDC less than 










2 years (b) 


67.8 ** 


75.5 


58.5 




Received AFDC 2 years 










or more (b) 


63.0 


69.9 


51.9 


60 2 


Household Composition 










Spouse present 


66.3 


76.3 


56.7 


64.6 


No spouse present, child present 


66.6 


73.0 


57.7 


64.5 


No spouse present, no child 










present 


59.9 *** 


67.4 ** 


53.3 


61.2 



SOURCE: MDRC calculations from Background Information Form responses and 
program enrollment data from the 16 SDAs. 

NOTES: Calculations for this table are based on data for all adult female treatment 
group members in the 18-month study sample. The total sample size may vary among 
characteristics because of missing data for some treatment group members. 

For each characteristic, within each service strategy, a chi-square test 
was applied to the difference in enrollment rates between the first subgroup 
listed and, separately, each of the remaining subgroups. Statistical significance 
levels are indicated as * = 10 percent; ** = 5 percent; *** « 1 percent. 

(a) The barriers to employment include: not having a high school diploma 
or GED at the time of random assignment; having worked fewer than 13 weeks 
during the 12 months prior to random assignment; and receiving AFDC or other 
cash assistance at the time of random assignment. 

(b) The AFDC receipt may not have been continuous. 
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TABLE B.7 



ENROLLMENT RATES FOR MALE YOUTH TREATMENT GROUP MEMBERS 
BY SELECTED BASELINE CHARACTERISTICS AND SERVICE STRATEGY ' 



Characteristic 
and Subgroup 



All Male Youth Treatment 
Group Members 

Age 
16-19 
20-21 

Ethnicity 
White, non-Hispanic 
Black, non-Hispanic 
Hispanic 
Other 

Education 
High school diploma or GED 
No high school diploma or GED 

Work Experience 
Worked 13 weeks or more 

in 12 months before assignment 
Worked fewer than 13 weeks 
in 12 months before assignment 

Public Assistance Receipt 
Not receiving AFDC or other cash 

assistance at assi gnment 
Receiving AFDC or other cash 
assistance at ass ignment 

Barriers to Employment (b) 
None 

1 
2 
3 



All 


Classroom 




\JIDCT 


Strategies 


Training 


OJT/JSA 


Services 


(*) 


(%) 


(%) 


V n } 


66.8 


74.8 


58.5 


ffl 7 


68.9 


75 7 


Dy.U 


70.2 


63.2 ** 


72.9 


57.9 


62.8 * 


68.7 


73.3 


62.7 


73.1 


59.7 *** 


67.6 


44.4 *** 


61.3 *** 


72.0 


84.0 ** 


55.2 


66.0 


80.0 


100.0 (a) 


50.0 


84.2 


62.2 


75.8 


59.2 


52.8 


70.3 *** 


75.6 


55.9 


73.8 *** 


64.7 


73.0 


57.1 


67.3 


69.3 * 


74.7 


60.8 


70.4 


65.9 


74.2 


58.7 


66.0 


66.5 


71.4 


55.1 


71.7 


61.2 


74.7 


58.6 


52.6 


66.4 * 


74.3 


58.7 


66.8 ** 


73.7 *** 


76.6 


58.7 


77.9 *** 


63.4 


69.6 


50.0 


64.7 
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TABLE B.7 (continued) 



All Classroom Other 

Characteristic Strategies Training OJT/JSA Services 

and Subgroup (%) (%) (%) (%) 

AFDC History 

Never received AFDC 66.6 74.8 57.9 67.8 
Received AFDC less than 

2 years (c) 75.0 71.4 75.0 100.0 (a) 

Received AFDC 2 years 

or more (c) 66.7 0.0 (a) 0.0 (a) 66.7 

Household Composition 

Spouse present 74.8 83.9 73.7 69.4 

No spouse present, child present 63.8 82.6 43.7 ** 57.9 
No spouse present, no child 

Present 65.4 ** 72.5 55.1 *** 68.2 



SOURCE: MDRC calculations from Background Information Form responses and 
program enrollment data from the 16 SDAs. 

NOTES: Calculations for this table are based on data for all male youth treatment 
group members in the 18-month study sample. The total sample size may vary among 
characteristics because of missing data for some treatment group members. 

For each characteristic, within each service strategy, a chi-square test 
was applied to the difference in enrollmeat rates between the first subgroup 
listed and, separately, each of the remaining subgroups. Statistical significance 
levels are indicated as * « 10 percent; ** « 5 percent; *** « i percent. 

(a) A chi-square test was inappropriate because of low expected cell frequencies. 

(b) The barriers to employment include: not having a high school diploma 
or GED at the time of random assignment; having worked fewer than 13 weeks 
during the 12 months prior to random assignment; and receiving AFDC or other 
cash assistance at the time of random assignment. 

(c) The AFDC receipt may not have been continuous. 
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TABLE B.8 



ENROLLMENT RATES FOR FEMALE YOUTH TREATMENT GROUP MEMBERS, 
BY SELECTED BASELINE CHARACTERISTICS AND SERVICE STRATEGY 



Characteristic 
and Subgroup 



All Female Youth Treatment 
Group Members 

Age 
16-19 
20-21 

Ethnicity 
White, non-Hispanic 
Black, non-Hispanic 
Hispanic 
Other 

Education 
High school diploma or GED 
No high school diploma or GED 

Work Experience 
Worked 13 weeks or more 

in 12 months before assignment 
Worked fewer than 13 weeks 

in 12 months before assignment 

Public Assistance Receipt 
Not receiving AFDC or other cash 

assistance at assignment 
Receiving AFDC or other cash 
assistance at assignment 

Barriers to Employment (a) 
None 
1 

2 
3 



All Classroom Other 
Strategies Training OJT/JSA Services 
(*) (%) (%) (%) 



65.5 



71.5 



57.5 



63.1 



65.6 72.0 55.8 63.7 

65.3 70.7 59.5 61.8 



67.0 




68.0 


61.9 


70.3 


56.9 


*** 


UJ.O 


40 1 ** 




77.7 


*** 


86.7 *** 




71 O 


73.3 




85 7 


SO 0 


/D.U 


60.9 




72.0 


59.1 


43.4 


70.9 


*** 


71.3 


55.7 


76.7 


66.6 




71.7 


63.3 


61.0 


64.8 




70.6 


50.0 *** 


64.8 


65.8 




73.2 


59.3 


61.5 


64.4 




67.4 * 


54.6 


65.1 


65.5 




76.0 


63.8 


49.5 


62.3 




71.2 


55.0 


55.6 


66.4 




68.4 * 


53.6 


69.1 


72.0 


* 


71.4 


48.1 


79.' 
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TABLE B.8 (continued) 



All Classroom Other 

Characteristic Strategies Training OJT/JSA Services 

and Subgroup (%) (%) (%) (%) 

AFDC History 

Never received AFDC 65.7 72. 1 58.2 63.2 
Received AFDC less than 

2 years (b) 65.6 71.7 60.3 59.8 
Received AFDC 2 years 

or more (b) 62.0 64.7 38.1 ♦ 70.0 

Household Composition 

Spouse present 70.1 78.9 56.1 70.0 

No spouse present, child present 65.9 71.8 55.6 64.4 
No spouse present, no child 

Present 63.4 ♦ 69.2 * 58.2 59.9 



SOURCE: MDRC calculations from Background Information Form responses and 
program enrollment data from the 16 SDAs. 

NOTES: Calculations for this table are based on data for all female youth treatment 
group members in the 18-month study sample. The total sample size may vary among 
characteristics because of missing data for some treatment group members. 

For each characteristic, within each service strategy, a chi-square test 
was applied to the difference in enrollment rates between the first subgroup 
listed and, separately, each of the remaining subgroups. Statistical significance 
levels are indicated as * « 10 percent; ** = 5 percent; *** = 1 percent. 

(a) The barriers to employment include: not having a high school diploma 
or GED at the time of random assignment; having worked fewer than 13 weeks 
during the 12 months prior to random assignment; and receiving AFDC or other 
cash assistance at the time of random assi gnm ent. 

(b) The AFDC receipt may not have been continuous. 
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TABLE B.9 



SERVICE RECEIPT RATES FOR ADULT ENROLLEES, 
BY NUMBER AND TYPE OF SERVICES RECEIVED AND BY TARGET GROUP 

All Adult Adult 
Number and Type of Adults Men Women 
Services Received (%) ($) (%) 

Ever Received 
Classroom training in 

occupational skills (CT-OS) 38.0 

On-the-job training (OJT) 22.2 

Job search assistance (JSA) 36.5 

Basic education (BE) 10.9 

Miscellaneous services (a) 24.7 



26.3 47.2 

25.5 19.5 

40.1 33.6 

9.1 12.3 

24.5 24.8 



Received 1 Service 
CT-OS only 
OJT only 
JSA only 
BE only 

Miscellaneous only (a) 



73.7 


79.1 


69.7 


25.5 


19.4 


30.4 


14.1 


16.4 


12.3 


19.2 


25.0 


14.7 


3.2 


3.4 


3.0 


11.7 


14.9 


9.3 



Received 2 or More Services 
CT-OS + OJT 
CT-OS + JSA 
CT-OS + BE 

CT-OS + Miscellaneous (a) 

OJT + JSA 
OJT + BE 

OJT + Miscellaneous (a) 
JSA + BE 

JSA + Miscellaneous (a) 
BE + Miscellaneous (a) 



26.3 


20.9 


30.3 


1.3 


1.2 


1.4 


7.2 


3.6 


10.0 


2.6 


1.5 


3.4 


5.6 


3.2 


7.5 


5.3 


6.9 


4.1 


0.8 


0.9 


0.7 


3.0 


2.9 


3.1 


4.0 


3.5 


4.4 


5.1 


4.8 


5.3 


3.7 


1.9 


5.1 



Sample Size 5,169 2,286 2,883 

SOURCE: MDRC calculations from Background Information Form responses and from program 
enrollment and participation data from the 16 SDAs. 

NOTES: Calculations for this table are based on data for all adult enroUees in the 18-month 
study sample. 

In the top panel, the total service receipt rate for each target grou j is over 100.0 percent 
because some enroUees received more than one service, in the bottom panel, the total service receipt 
rate for each target group is over 26.3, 20.9, and 30.3 percent, respectively, because some enroUees 
received more than two services and are included in each combination of services they received. 

Tests of statistical significance were not performed. 

(a) Miscellaneous program services include such activities as assessment, job-readiness 
preparation, vocational exploration, job shadowing, and tryout employment In this table, 
miscellaneous program services also include work experience, which is identified as a separate 
program service in other reports on the National JTPA Study. 
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TABLE B.10 



SERVICE RECEIPT RATES FOR OUT-OF-SCHOOL YOUTH ENROLLEES, 
BY NUMBER AND TYPE OF SERVICES RECEIVED AND BY TARGET GROUP 





All 

All 


Male 


Female 


isumoer ana lype or 


Youths 


Youths 


Youths 


Services Received 




(%) 


(%) 


Ever Received 








Classroom training in 








occupational skills (CT-OS) 


36.4 


28.1 


43.1 


On-the-job training (OJT) 


16.5 


19.2 


14.3 


Job search assistance (JSA) 


35.3 


36.3 


34.5 


Basic education (BE) 


27.2 


26.5 


27.9 


Miscellaneous services (a) 


29.1 


31.9 


26.8 


Received 1 Service 


65.0 


66.6 


63.7 


rT-OC nnlv 

w ± V/u UIUV 


IT < 


10.5 


23.1 


OJT nnlv 
v/ j x mu v 


8.9 


10.0 


8.0 


TC A nnlt/ 
J OA UEUjr 


1A t\ 

10.9 


13.2 


9.0 


nc* oniy 


11.9 


12.9 


11. 1 


Miscellaneous only (a) 


IS.8 


20.0 


12.5 


ASvcivou z. or iviorc oervices 


K A 

JD.U 


33.4 


36.3 


W A V^»J » J J. 


i n 


1.1 


u.y 




10 o 


ill 
11.1 


14.5 


CT-OS + BE 


8.2 


7.8 


8.S 






A 1 
•1.1 


4./ 


OJT + JSA 


5.5 


7.4 


4.0 




0.7 


1.1 


0.3 


OJT + Miscellaneous (a) 


1.6 


1.3 


1.9 


JSA + BE 


9.1 


7.2 


10.6 


JSA + Miscellaneous (a) 


5.4 


5.0 


5.7 


BE + Miscellaneous (a) 


5.7 


4.9 


6.3 


Sample Size 


2,147 


959 


1,188 



SOURCE: MDRC calculations from Background Information Form responses and from program 
enrollment and participation data from the 16 SDAs. 

*\ 

NOTES: Calculations for this table are based on data for all out-of-school youth enrollees 
in the 18-month study sample. 

In the top panel, the total service receipt rate for each target group is over 100.0 percent 
because some enrollees received more than one service. In the bottom panel, the total service receipt 
rate for each target group is over 26.3, 20.9, and 30.3 percent, respectively, because some enrollees 
received more than two services and are included in each combination of services they received. 

Tests of statistical significance were not performed. 

(a) Miscellaneous program services include such activities as assessment, job-readiness 
preparation, vocational exploration, job shadowing, and try out employment. In this table, 
miscellaneous program services also include work experience, which is identified as a separate 
program service in other reports on the National JTPA Study. 
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TABLE B.ll 



SERVICE RECEIPT RATES FOR ADULT MALE ENROLLEES, 
BY SELECTED BASELINE CHARACTERISTICS 



Characteristic 



Sample 

Size 



Non- Misc. 
Training JSA Non-Training 
CT-OS (a) OJT (b) Education (c) Only Services (d) 



All Adult Male FnmlWc 


O OCA 


26.3 




24.3 




7.2 




25.0 




17.2 


Age 






















22-29 


1,039 


27.8 




26.0 




7.2 




20.9 




18.1 


<JA A A 

30-44 


969 


25.4 




23.4 




7.5 




26.8 


*** 


16.8 


45 and over 


278 


24.1 




21.2 




5.8 




33.8 


*** 


15 1 


Ethnicity 






















White, non-Hispanic 


1,319 


18.5 




30.6 




ft 7 




OA. 0 




17 A 


Black 9 non-Hispanic 


627 


40.8 


*** 


15.2 


*** 


1.9 


*** 


27.0 




15.2 


Hispanic 


237 


23.2 


* 


18.1 


*** 


14.3 


*** 


23.6 




20.7 


Other 


103 


45.6 


*** 


14.6 


*** 


2.9 


* 


17.5 


* 


19.4 


Education 






















High school diploma or GED 


1,501 


29.3 




24.2 




3.5 




25.8 




17.1 


No high school diploma or GED 


637 


19.3 


*** 


25.1 




16.0 


*** 


22.6 




17.0 


Work Experience 






















Worked 13 weeks or more 






















in 12 months before assignment 


1,226 


26.1 




26.5 




5.6 




25.6 




16.2 


Worked fewer than 13 weeks 






















in 12 months before assignment 


846 


27.0 




21.4 


*** 


9.1 


*** 


23.3 




19.3 * 


Public Assistance Receipt 






















Not receiving AFDC or other cash 






















assistance at assignment 


1,698 


26.9 




25.4 




4.7 




25.8 




17.1 


Receiving AFDC or other cash 






















assistance at assignment 


321 


23.4 




19.6 


*» 


21.2 


*** 


21.5 




14.3 


Barriers to Employment (e) 






















None 


949 


28.7 




25.5 




2.0 




26.6 




17.3 


1 


906 


26.5 




24.8 




6.8 


*** 


25.4 




16.4 


2 


356 


21.1 




22.2 




16.3 


*** 


22.8 




17.7 


3 


62 


19.4 




11.3 


»» 


37.1 


*** 


9.7 


*** 


22.6 
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TABLE B.ll (continued) 



Characteristic 
and Subgroup 



Non- Misc. 

Training JSA Non-Training 

Sample CT-OS (a) OJT (b) Education (c) Only Services (d) 

Size (%) (%) (%) (%) (%) 



AFDC History 
Never received AFDC 
Received AFDC less than 
2 years (f) 

Received AFDC 2 years 
or mere (f) 



2,067 26.9 25.1 6.0 24.7 17.4 

159 21.4 16.4 ** l&/> *** 28.3 15.7 
37 18.9 24.3 13.5 * 29.7 13.5 



Household Composition 
Spouse present 800 
No spouse present, child present 167 
No spouse present, no child 
present 1,121 



22.4 
26.3 



25.0 
29.9 



9.0 
7.8 



28.0 
21.0 * 



15.6 
15.0 



28.9 *** 23.6 



5.7 *** 23.3 ** 18.5 



SOURCE: MDRC calculations from Background Information Form responses and from program enrollment 
and participation data from the 16 SDAs. 

NOTES: Calculations for this table are based on data for all adult male enrollees in the 18-month study 
sample. The total sample size may vary among characteristics because of missing data for some enrollees. 

The program service categories in this table are mutually exclusive; see notes a, b, c, and d below. 

For each characteristic, within each program service category, a chi-square test was applied to the 
difference in service receipt rates between the first subgroup listed and, separately, each of the remaining 
subgroups. Statistical significance levels are indicated as * * 10 percent; ** « 5 percent; *** = 1 percent. 

(a) The CT-OS service category includes enrollees who received CT-OS only, or CT-OS in 
combination with OJT, basic education, job search assistance, or miscellaneous program services. 

(b) The OJT service category includes enrollees who received OJT only, or OJT in combination 
with basic education, job search assistance, or miscellaneous program services. 

(c) The non-training education service category includes enrollees who received basic education 
only, or basic education in combination with job search assistance or miscellaneous program services. 

(d) The miscellaneous non-training services category includes enrollees who received miscellane- 
ous program services only, or miscellaneous program services in combination with job search assistance. 
Miscellaneous program services include such activities as assessment, job-readiness preparation, vocational 
exploration, job shadowing, and tryout employment. In this table, miscellaneous program services also 
include work experience, which is identified as a separate program service in other reports on the National 
JTPA Study. 

(e) The barriers to employment include: not having a high school diploma or GED at the time of 
random assignment; having worked fewer than 13 weeks during the 12 months prior to random assignment; 
and receiving AFDC or other cash assistance at the time of random assignment. 

(0 The AFDC receipt may not have been continuous. 
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TABLE B. 12 

SERVICE RECEIPT RATES FOR ADULT FEMALE ENROLLEES, 
BY SELECTED BASELINE CHARACTERISTICS 



Characteristic 
and Subgroup 



All Adult Female Enrollees 



Non- 
Training JSA 
Sample CT-OS (a) OCT (b) Education (c) Only 
Size (%) (%) (%) ( %) 



2,883 



47.2 



18.1 



8.4 



14.7 



Misc. 
Non-Training 
Services (d) 
(*) 



11.6 



Age 
22-29 
30-44 
45 and ever 

Ethnicity 
White, non-Hispanic 
Black, non-Hispanic 
Hispanic 
Other 

Education 
High school diploma or GED 
No high school diploma or GED 

Work Experience 
Worked 13 weeks or more 

in 12 months before assignment 
Worked fewer than 13 weeks 

in 12 months before assignment 

Public Assistance Receipt 
Not receiving AFDC or other cash 

assistance at assignment 
Receiving AFDC or other cash 

assistance at assignment 

Barriers to Employment (e) 



1,266 
1,270 
347 



48.4 

49.7 
34.0 



19.4 
16.0 
21.0 



8.9 
8.7 
5.5 



12.7 
13.9 
24.8 



10.6 
11.7 
14.7 



1,622 


40.4 




22.1 




9.4 




16.0 




12.1 


805 


58.0 




16.0 




3.4 




14.0 




8.6 


352 


49.1 




6.3 




16.8 




11.4 


•* 


16.5 


104 


64.4 




11.5 


•* 


4.8 




10.6 




8.7 


1,990 


50.4 




18.2 




4.4 




16.1 




10.9 


705 


38.6 




18.6 




19.4 




11.2 




12.2 


1,202 


42.9 




20.8 




6.4 




18.1 




11.7 


1,332 


51.9 




15.0 




9.8 




11.6 




11.7 


1,589 


43.3 




22.1 




5.5 




18.3 




10.8 


1,084 


54.2 




12.4 


*** 


12.7 




8.6 




12.2 



None 


879 


43.1 


22.5 




3.3 




20.3 




10.8 


1 


1,046 


47.0 • 


18.8 


•* 


6.5 




16.3 


•* 


11.4 


2 


736 


52.7 


14.1 


•** 


12.4 




8.6 




12.2 


3 


201 


46.8 


10.0 




25.9 




5.0 




12.4 
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TABLE B.12 (continued) 



Non- Misc. 

Training JSA Noo-Tnuni^ 

Characteristic Sample CT-OS (a) OJT (b) Education (c) Only Services (d) 

and Subgroup Size (%) (%) (%) (%) (%) 



AFDC History 



Never received AFDC 


1,394 


41.5 




20.5 


7.0 


20.0 




11.0 


Received AFDC less than 


















2 years (f) 


678 


49.4 


•** 


19.2 


8.6 


11.1 


•** 


11.8 


Received AFDC 2 years 


















or more (f) 


762 


55.8 


**• 


13.1 *** 


11.0 *** 


8.0 


•** 


12.1 


[ousehold Composition 


















Spouse present 


610 


43.1 




21.3 


9.8 


15.2 




10.5 


No spouse present, child present 


1,391 


52.3 


•** 


16.6 ** 


8.3 


11.6 


•* 


11.2 


No spouse present, no child 


















present 


512 


36.1 


•* 


20.5 


7.0 * 


22.9 


•** 


13.5 



SOURCE: MDRC calculations from Background Information Form responses and from program enrollment 
and participation data from the 16 SDAs. 

NOTES: Calculations for this table are based on data for all adult female enrollees in the 18-month study 
sample. The total sample size may vary among characteristics because of missing data for some enrollees. 

The program service categories in this table are mutually exclusive; see notes a, b, c, and d below. 

For each characteristic, within each program service category, a chi-square test was applied to the 
difference in service receipt rates between the first subgroup listed and, separately, each of the remaining 
subgroups. Statistical significance levels are indicated as * « 10 percent; ♦* « 5 percent; *** = 1 percent. 

(a) The CT-OS service category includes enrollees who received CT-OS only, or CT-OS in 
combination with OJT, basic education, job search assistance, or miscellaneous program services. 

(b) The OJT service category includes enrollees who received OJT only, or OJT in combination 
with basic education, job search assistance, or miscellaneous program services. 

(c) The non-training education service category includes enrollees who received basic education 
only, or basic education in combination with job search assistance or miscellaneous program services. 

(d) The miscellaneous non-training services category includes enrollees who received miscellane- 
ous program services only, or miscellaneous program services in combination with job search assistance. 
Miscellaneous program services include such activities as assessment, job-readmess preparation, vocational 
exploration, job shadowing, and tryout employment. In this table, miscellaneous program services also 
include work experience, which is identified as a separate program service in other reports on the National 
JTPA Study. 

(e) The barriers to employment include: not having a high school diploma or GED at the time of 
random assignment; having workM fewer than 13 weeks during the 12 months prior to random assignment; 
and receiving AFDC or other cash assistance at the time of random assignment. 

(f) The AFDC receipt may not have been continuous. 
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TABLE B.13 



SERVICE RECEIPT RATES FOR MALE YOUTH ENROLLEES, 
BY SELECTED BASELINE CHARACTERISTICS 



Characteristic 
and Subgroup 



Nan- Misc. 

Training JSA Non-Training 

Sample CT-OS (a) OJT (b) Education (c) Only Services (d) 

Si» (%) (%) (%) (%) 



All Male Youth Enrollees 


959 


28.1 


18.0 




17.9 




1^ 0 




99 7 




Age 






















16-19 


615 


29.9 


14.1 




21.0 




13.0 




22.0 




20-21 


344 


24 7 * 




*** 


19 < 


*** 


IJ. / 




24.1 




Ethnicity 






















White. non-Hismnic 




91 O 


tA < 




13.0 




16.2 




24.3 




Black, non-Hispanic 


253 


26.1 


6.3 


*** 


35.2 


*** 


10.7 


»* 


21.7 




Hispanic 


152 


52.0 *** 


16.4 


+ + 


7.2 


* 


7.2 


»** 


17.1 




Other 


24 


33.3 


8.3 


* 


12 5 




19 ^ 




J J .J 




Education 






















High school diploma or GED 


351 


32.2 


29.1 




2.0 




17.4 




10 A 




No high school diploma or GED 


572 


26.2 * 


10.8 


*** 


28.0 




10.1 


*** 


24.8 




Work Experience 






















Worked 13 weeks or more 






















in 12 months before assignment 


449 


27.4 


24.3 




12.0 




15.1 




21.2 




Worked fewer than 13 weeks 






















in 12 months before assignment 


427 


28.3 


12.2 


*** 


23.4 


»** 


11.5 




24.6 




Public Assistance Receipt 






















Not receiving AFDC or other cash 






















assistance at assignment 


738 


28.5 


20.6 




15.4 




14.1 




21.4 




Receiving AFDC or other cash 






















assistance at assignment 


105 


21.0 


5.7 


»»» 


25.7 


**» 


13.3 




34.3 


*** 


Barriers to Employment (e) 






















None 


227 


30.8 


34.4 




1.8 




16.7 




16.3 




1 


403 


28.5 


17.1 


*** 


16.4 


*** 


15.1 




22.8 




2 


283 


26.1 


8.5 


*** 


30.0 


*** 


8.5 


+++ 


26.9 


*** 


3 


45 


22.2 


2.2 




37.8 


*** 


8.9 




28.9 


•* 
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TABLE B.13 (continued) 



Non- Misc. 

Training JSA Non-Training 

Characteristic Sample CT-OS (a) OJT (b) Education (c) Only Services (d) 

and Subgroup Size (%) (%) (%) (%) (%) 

AFDC History 



Never received AFDC 


937 


28.5 




18.1 




17.9 


13.2 


22.2 


Received AFDC less than 


















2 years (f) 


15 


0.0 


(g) 


13.3 




26.7 


13.3 


46.7 ** 


Received AFDC 2 years 


















or more (f) 


2 


0.0 


(g) 


0.0 


(g) 


0.0 (g) 


0.0 (g) 


100.0 (g) 


[ousebold Composition 


















Spouse present 


no 


17.3 




31.8 




12.7 


15.5 


22.7 


No spouse present, child present 


37 


29.7 




13.5 


** 


24.3 * 


16.2 


16.2 


No spouse present, no child 


















present 


758 


30.1 


*** 


16.8 




17.2 


12.9 


23.1 



SOURCE: MDRC calculations from Background Information Form responses and from program enrollment 
and participation data from the 16 SDAs. 

NOTES: Calculations for this table are baaed on data for all male youth enrollees in the 18-month study 
sample. The total sample size may vary among characteristics because of missing data for some enrollees. 

The program service categories in this table are mutually exclusive; see notes a, b, c, and d below. 

For each characteristic, within each program service category, a chi-square test was applied to the 
difference in service receipt rates between the first subgroup listed and, separately, each of the remaining 
subgroups. Statistical significance levels are indicated as * «• 10 percent; ** = 5 percent; *** = 1 percent. 

(a) The CT-OS service category includes enrollees who received CT-OS only, or CT-OS in 
combination with OJT, basic education, job search assistance, or miscellaneous program services. 

(b) The OJT service category includes enrollees who received OJT only, or OJT in combination 
with basic education, job search assistance, or miscellaneous program services. 

(c) The non-training education service category includes enrollees who received basic education 
only, or basic education in combination with job search assistance or miscellaneous program services. 

(d) The miscellaneous noc -training services category includes enrollees who received miscellane- 
ous program services only, or miscellaneous program services in combination with job search assistance. 
Miscellaneous program services include such activities as assessment, job-readiness preparation, vocational 
exploration, job shadowing, and tryout employment. In this table, miscellaneous program services also 
include work experience, which is identified as a separate program service in other reports on the National 
JTPA Study. 

(e) The barriers to employment include: not having a high school diploma or GED at the time of 
random assignment; having worked fewer than 13 weeks during the 12 months prior to random assignment; 
and receiving AFDC or other cash assistance at the time of random assignment. 

(0 The AFDC receipt may not have been continuous. 

(g) A chi-square test was inappropriate because of low expected cell frequencies. 
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TABLE B. 14 



SERVICE RECEIPT RATES FOR FEMALE YOUTH ENROLLEES, 
BY SELECTED BASELINE CHARACTERISTICS 



Characteristic 
andSubg 

All Female Youth Enrollees 

Age 
16-19 
20-21 

Ethnicity 
White, non-Hispanic 
Black, non-Hispanic 
Hispanic 
Other 

Education 
High school diploma or GED 
No high school diploma or GED 

Work Experience 
Worked 13 weeks or more 

in 12 months before assignment 
Worked fewer than 13 weeks 
in 12 months before assignment 

Public Assistance Receipt 
Not receiving AFDC or other cash 

assistance at assignment 
Receiving AFDC or other cash 

assistance at assignment 

Barriers to Employment (f) 
None 
1 

2 
3 



Sample 
Size 



NoQ - Misc. 

Training JSA Non-Training 

CT-OS(a) OJT (b) Education (c) Only Services (d) 
A*) 1%) (%) 



(%) 



1,188 


43.1 


13.4 




19.2 




9.0 


15.3 


707 


40.7 


11.0 




23.1 




8.6 


LO.D 


481 


46.6 


16.8 




13.5 




9.6 


13.5 


608 


36.8 


18.6 




18.6 




10.4 


15.6 


335 


38.2 


7.5 




29.3 




8.4 


16.7 


223 


66.8 


7.2 




6.7 


*** 


7.2 


19 1 


22 


50.0 


22.7 




9.1 




0.0 (e) 


18.2 


534 


51.9 


20.6 




3.0 




10.5 


14.0 


601 


34.8 


6.8 




33.4 




7.5 • 


17.5 


428 


44 2 


17 1 




1 ft f\ 

10.0 




13.1 


15.4 


637 


47.7 


10.5 






*** 




15.1 


741 


43.3 


14.7 




16.S 




11.2 


14.3 


353 


42.8 


11.3 




24.6 




5.9 **• 


15.3 


272 


47.1 


21.0 




4.0 




14.3 


13.6 


395 


47.3 


15.7 


* 


10.6 


*** 


10.6 


15.7 


355 


39.4 • 


9.6 




30.7 


*** 


5.4 •** 


14.9 


162 


34.0 


3.7 




40.1 


*** 


4.3 


17.9 
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TABLE B.14 (continued) 



Non- Misc. 

Training JSA Non-Training 

Sample CT-OS (a) OJT (b) Education (c) Only Services (d) 

Size (%) (%) (%) (%) (%) 



Characteristic 

and Subgroup 

AFDC History 
Never received AFDC 
Received AFDC less than 
2 years (g) 

Received AFDC 2 years 
or more (g) 

Household Composition 
Spouse present 

No spouse present, child present 
No spouse present, no child 
present 



846 42.8 13.0 

252 40.9 15.1 

80 53.7 • 11.3 

138 44.9 11.6 

400 44.3 14.2 

558 42.3 14.0 



17.6 10.4 16.2 

23.8 ** 6.0 ** 14.3 
21.3 5.0 8.8 * 

23.9 8.0 11.6 
20.5 8.0 13.0 

15.1 ** 10.2 18.5 * 



SOURCE: MDRC calculations from Background Information Form responses and from program enrollment 
and participation data from the 16 SDAs. 

NOTES: Calculations for this table are based on data for all female youth enroUees in the 18-month study 
sample. The total sample size may vary among characteristics because of missing data for some enrollees. 

The program service categories in this table are mutually exclusive; see notes a, b, c, and d below. 

For each characteristic, within each program service category, a chi-square test was applied to the 
difference in service receipt rates between the first subgroup listed and, separately, each of the remaining 
subgroups. Statistical significance levels are indicated as * « 10 percent; ** «= 5 percent; *** = 1 percent. 

(a) The CT-OS service category includes enrollees who received CT-OS only, or CT-OS in 
combination with OJT, basic education, job search assistance, or miscellaneous program services. 

(b) The OJT service category includes enroUees who received OJT only, or OJT in combination 
with basic education, job search assistance, or miscellaneous program services. 

(c) The non-training education service category includes enrollees who received basic education 
only, or basic education in combination with job search assistance or miscellaneous program services. 

(d) The miscellaneous non-training services category includes enrollees who received miscellane- 
ous program services only, or miscellaneous program services in combination with job search assistance. 
Miscellaneous program services include such activities as assessment, job-readiness preparation, vocational 
exploration, job shadowing, and tiyout employment. In this table, miscellaneous program services also 
include work experience, which is identified as a separate program service in other reports on the National 
JTPA Study. 

(e) A chi-square test was inappropriate because of low expected cell frequencies. 

(f) The barriers to employment include: not having a high school diploma or GED at the time of 
random assignment; having worked fewer than 13 weeks during the 12 months prior to random assignment; 
and receiving AFDC or other cash assistance at the time of random assi gnment 

(g) The AFJC receipt may not have been continuous. 
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TABLE B. 15 



SERVICE RECEIPT RATES FOR ADULT MALE ENROLLEES, 
BY EMPLOYMENT BARRIER SUBGROUP AND ETHNICITY 



9 

ERIC 



Employment Barrier Subgroup (a) Sample CT-OS (b) 
and Ethnicity Size (%) 



Non- 
Training 
OJT (c) Education (d) 
(%) 



JSA 
Only 



Misc. 
Non-Training 
Services (e) 



No Employment Barriers 




















White noo— T-Tisnanir 


Dot 


xy. l 




31.5 




1.7 




26.6 


21.1 


Black, non-Hispanic 


259 


46.3 


*** 


15.4 


*** 


0.8 




27 8 

Ami • O 


0 7 **• 

y. / 


Hispanic 


75 


33.3 


*** 


16.0 


*»* 


9.3 


•** 


24 0 


n "k 


1 Employment Barrier 




















Worked fewer than 13 weeks 




















in 12 months before assignment 




















White, non-Hispanic 


226 


25.7 




28.3 




5.3 




25.2 


15.5 


Black, non-Hispanic 


183 


38.3 


*** 


14.8 


*** 


3.8 




23.5 


19.7 


Hispanic 


58 


24.1 




19.0 




12.1 


* 


27.6 


17.2 


No high school diploma or GED 




















at assignment 




















White, non-Hispanic 


197 


13.2 




38.6 




7.6 




27.9 


12.7 


Black, non-Hispanic 


57 


45.6 


*** 


12.3 


*** 


3.5 




22.8 


15.8 


Hispanic 


50 


12.0 




24.0 


* 


20.0 


«» 


22.0 


22.0 * 


Receiving AFDC or other cash 




















assistance at assignment 




















White, non-Hispanic 


64 


21.9 




25.0 




9.4 




26.6 


17.2 


Black, non-Hispanic 


28 


32.1 




17.9 




0.0 


(0 


39.3 


10.7 


Hispanic 


2 


0.0 


(f) 


50.0 




50.0 


0.0 (0 


0.0 (0 



2 Employment Barriers 

Worked fewer than 13 weeks 
in 12 months before assignment 
and no high school diploma or GED 
at assignment 

White, non-Hispanic 
Black, non-Hispanic 
Hispanic 

Worked fewer than 13 weeks 
in 12 months before assignment 
and receiving AFDC or other cash 
assistance at assignment 

White, non-Hispanic 

Black, non-Hispanic 

Hispanic 



91 


16.5 


29.7 


13.2 


20.9 


19.8 


43 


16.3 


20.9 


2.3 * 


34.9 * 


25.6 


41 


12.2 


14.6 * 


17.1 


24.4 


31.7 



56 


21.4 


32.1 


10.7 


21.4 


14.3 


31 


45.2 ** 


12.9 * 


0.0 (f) 


29.0 


12.9 


5 


40.0 


0.0 (0 


20.0 


20.0 


20.0 
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TABLE B.15 (continued) 



Noo- Misc. 

Training JSA Non-Training 

Employment Barrier Subgroup (a) Sample CT-OS (b) OJT (c) Education (d) Only Services (e) 

and Ethnicity Size (%) (%) (%) (%) (%) 

No high school diploma or GED 
at assignment and receiving 
AFDC or other cash assitance 
at assignment 



White, non-Hispanic 


54 


7.4 




20.4 


53.7 




13.0 




5.6 


Black, non-Hispanic 


7 


42.9 


** 


14.3 


0.0 


(f) 


42.9 


* 


0.0 


Hispanic 


2 


50.0 


* 


0.0 (f) 


50.0 




0.0 


(0 


0.0 


All 3 Emntovment Jarriers 




















White, non-Hispanic 


37 


5.4 




10.8 


62.2 




10.8 




10.8 


Black, non-Hispanic 


13 


38.5 




7.7 


0.0 


(f) 


15.4 




38.5 


Hispanic 


4 


50.0 


** 


25.0 


0.0 


(f) 


0.0 


(f) 


25.0 



SOURCE: MDRC calculations from Background Information Form responses and from program enrollment 
and participation data from the 16 SDAs. 

NOTES: Calculations for this table are based on data for all white, black, and Hispanic adult male enrollees 
in the 18-month study sample. Enrollees from other ethnic groups were not included because of small 
sample sizes. 

The program service categories in this table are mutually exclusive; see notes b, c, d, and e below. 

For each employment barrier subgroup, within each program service category, a chi-square test 
was applied to the difference in service receipt rates between white enrollees and black enrollees and 
between white enrollees and Hispanic enrollees. Statistical significance levels are indicated as 
* «= 10 percent; « 5 percent; « 1 percent 

(a) The barriers to employment include: not having a high school diploma or GED at the time of 
random assignment; having worked fewer than 13 weeks during die 12 months prior to random assignment; 
and receiving AFDC or other cash assistance at the time of random assignment. 

(b) The CT-OS service category includes enrollees who received CT-OS only, or CT-OS in 
combination with OJT, basic education, job search assistance, or miscellaneous program services. 

(c) The OJT service category includes enrollees who received OJT only, or OJT in combination 
' with basic education, job search assistance, cr miscellaneous program services. 

(d) The non-training education service category includes enrollees who received basic education 
only, or basic education in combination with job search assistance or miscellaneous program services. 

(e) The miscellaneous non-training services category includes enrollees who received miscellane- 
ous program services only, or miscellaneous program services in combination with job search assistance. 
Miscellaneous program services include such activities as assessment, job-readiness preparation, vocational 
exploration, job shadowing, and try out employment. In this table, miscellaneous program services also 
include work experience, which is identified as a separate program service in other reports on the National 
JTPA Study. 

(f) A chi-square test was inappropriate because of low expected cell frequencies. 
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TABLE B. 16 



SERVICE RECEIPT RATES FOR ADULT FEMALE ENROLLEES, 
BY EMPLOYMENT BARRIER SUBGROUP AND ETHNICITY 



Employment Barrier Subgroup (a) Sample 
and Ethnicity ^izt 



Non- Misc. 
Training JSA Non-Training 
CT-OS(b) OJT(c) Education (d) Only Service* (e) 
<*> <*> <*) (%) (%) 



No Esnafeaaaa Bsnaa 

White, non-Hi tpanic 
Black, non-Hispanic 
Hispanic 



575 


40.9 


24.7 


2.6 


20.3 


192 


45.3 


22.9 


1.6 


21.9 


83 


51.8 * 


10.8 *** 


12.0 *** 


16.9 



11.5 
8.3 
8.4 



ERIC 



Worked fewer than 13 weeks 
in 12 months before assi gnmen t 

White, non-Hispanic 

Black, non-Hispanic 

Hispanic 

No high school diploma or GED 
at assignment 

White, non-Hispanic 

Black, non-Hispanic 

Hispanic 

Receiving AFDC or other cash 
assistance at assignment 

White, non-Hispanic 

Black, non-Hispanic 

Hispanic 

2 Employment Harpers 

Worked fewer than 13 weeks 
in 12 months before assi gnmen t 
and no high school diploma or GED 
at assignment 

White, non-Hispanic 
Black, non-Hispanic 
Hispanic 

Worked fewer than 13 weeks 
in 12 months before assi gnment 
and receiving AFDC or other cash 
assistance at assi gnmen t 

White, non-Hispanic 

Black, non-Hispanic 

Hispanic 



323 


45.2 


18.6 


4.0 


18.9 


13.3 


99 


47.5 


24.2 


1.0 


21.2 


6.1 * 


61 


57.4 * 


4.9 ** 


8.2 


18.0 


11.5 



139 


30.2 


30.9 


6.5 


18.0 


14.4 


41 


46.3 * 


26.8 


7.3 


12.2 


7.3 


43 


27.9 


0.0 (f) 


44.2 *** 


14.0 


14.0 



142 


47.2 


20.4 


7.0 


13.4 


12.0 


124 


61.3 ** 


15.3 


1.6 * 


12.1 


9.7 


28 


50.0 


7.1 


14.3 


10.7 


17.9 



104 


30.8 


33.7 


11.5 




11.5 


12.5 


19 


42.1 


1S.8 


0.0 


(0 


31.6 ** 


10.5 


24 


33.3 


12.5 * 


41.7 


••• 


4.2 


8.3 



175 


51.4 


13.7 


15.4 


6.9 


12.6 


216 


71.8 *** 


8.3 * 


3.2 *** 


7.9 


8.8 


63 


63.5 


6.3 


3.2 ** 


3.2 


23.8 ** 



(continued) 
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TABLE B.16 (continued) 



9 

ERIC 



Nod- Misc. 

Training JSA Non-Training 

Employment Barrier Subgroup (a) Sample CT-OS (b) OJT (c) Education (d) Only Services (e) 

and Ethnicity Size (*) (%) (*) (*) (*) 

No high school diploma or GED 
at assignment and receiving 
AFDC or other cash assitance 
at assignment 

White, non-Hispanic 

Black, non-Hispanic 

Hispanic 



All 3 Employment Barriers 

White, non-Hispanic 
Black, non-Hispanic 
Hispanic 



64 


18.8 




18.8 


39.1 




14.1 


VA 


36 


63.9 




8.3 


11.1 


*** 


5.6 


11.1 


12 


33.3 




8.3 


16.7 




8.3 


33.3 


92 


29.3 




12.0 


43.5 




4.3 


10.9 


70 


68.6 




10.0 


10.0 




4.3 


7.1 


33 


48.5 


•* 


0.0 (f) 


15.2 




6.1 


30.3 



** 



SOURCE: MDRC calculations from Background Information Form responses and from program enrollment 
and participation data from the 16 SDAs. 

NOTES: Calculations for this table are based on data for all white, black, and Hispanic adult female 
enroll ecs in the 18-month study sample. Enrollees from other ethnic groups were not included because of 
small sample sizes. 

The program service categories in this table are mutually exclusive; see notes b, c, d, and e below. 

For each employment barrier subgroup, within each program service category, a chi-square test 
was applied to the difference in service receipt rates between white enrollees and black enrollees and 
between white enrollees and Hispanic enrollees. Statistical significance levels are indicated as 
* ~ 10 percent; ** * 5 percent; *** « 1 percent. 

(a) The barriers to employment include: not having a high school diploma or GED at the time of 
random assignment; having worked fewer than 13 weeks during the 12 months prior to random assignment; 
and receiving AFDC or other cash assistance at the time of random assignment. 

(b) The CT-OS service category includes enrollees whe received CT-OS only, or CT-OS in 
combination with OJT, basic education, job search assistance, or miscellaneous program services. 

(c) The OJT service category includes enrollees who received OJT only, or OJT in combination 
with basic 'education, job search assistance, or miscellaneous program services. 

(d) The non-training education service category includes enrollees who received basic education 
only, or basic education in combination with job search assistance or miscellaneous program services. 

(e) The miscellaneous non-training services category includes enrollees who received miscellane- 
ous program services only, or miscellaneous program services in combination with job search assistance. 
Miscellaneous program services include such activities as assessment, job-readiness preparation, vocational 
exploration, job shadowing, and try out employment In this table, miscellaneous program services also 
include work experience, which is identified as a separate program service in other reports on the National 
JTPA Study. 

(f) A chi-square test was inappropriate because of low expected cell frequencies. 
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TABLE B. 17 



SERVICE RECEIPT RATES FOR MALE YOUTH ENROLLEES, 
BY EMPLOYMENT BARRIER SUBGROUP AND ETHNICITY 



Employment Barrier Subgroup (a) Sample CT-OS (b) 
and Ethnicity Size (%) 



Non- 
Training JSA 
OJT (c) Education (d) Only 
(*) (*) (%) 



Misc. 
Non-Training 
Services (e) 

(*) 



No Employment Barriers 

White, non-Hispanic 
Black, non-Hispanic 
Hispanic 

1 Employment Barrier 

Worked fewer than 13 weeks 
in 12 months before assi gnmen t 

White, non-Hispanic 

Black, non-Hispanic 

Hispanic 

No high school diploma or GED 
at assignment 

White, non-Hispanic 

Black, non-Hispanic 

Hispanic 

Receiving AFDC or other cash 
assistance at assignment 

White, non-Hispanic 

Black, non-Hispanic 

Hispanic 

2 Employment Barriers 

Worked fewer than 13 weeks 
in 12 months before assignment 
and no high school diploma or GED 
at assignment 

White, non-Hispanic 
Black, non-Hispanic 
Hispanic 

Worked fewer than 13 weeks 
in 12 months before assignment 
and receiving AFDC or other cash 
assistance at assignment 

White, non-Hispanic 

Black, non-Hispanic 

Hispanic 



164 


25.6 


38.4 


1.8 


17.1 


17.1 


28 


46.4 ** 


17.9 ** 


0.0 (f) 


17.9 


17.9 


32 


43.7 ** 


31.3 


3.1 


9.4 


12.5 



69 


18.8 


30.4 


4.3 


18.8 


27.5 


31 


35.5 * 


9.7 ** 


9.7 


22.6 


22.6 


27 


59.3 *** 


18.5 


3.7 


14.8 


3.7 



141 


26.2 


18.4 


17.7 


14.9 


22.7 


75 


20.0 


6.7 ** 


40.0 *** 


12.0 


21.3 


33 


48.5 ** 


15.2 


6.1 


6.1 


24.2 



11 


9.1 


9.1 


9.1 




9.1 


63.6 


3 


33.3 


0.0 (f) 


0.0 


(f) 


66.7 * 


0.0 


3 


33.3 


33.3 


0.0 


(f) 


33.3 


0.0 



95 


18.9 


14.7 


23.2 


16.8 




26.3 


93 


21.5 


3.2 ** 


50.5 *** 


2.2 


*** 


22.6 


43 


55.8 *** 


9.3 


11.6 


0.0 


(f) 


23.3 



6 

2 
2 



16.7 
50.0 
50.0 



0.0 

0.0 (f) 
0.0 (f) 



0.0 
0.0 
0.0 



(0 
(f) 



33.3 
0.0 
50.0 



(0 



(0 

(f) 



50.0 
50.0 
0.0 (f) 



(continued) 
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TABLE B.17 (continued) 



Non- Misc. 
Training JSA Non-Training 

Employment Barrier Subgroup (a) Sample CT-OS (b) OJT (c) Education (d) Only Services (e) 



and Ethnicity 


Size 


(%) 




(%) 






(%) 




(*) 


No high school diploma or GED 




















at assignment and receiving 




















AFDC or other cash assitance 




















at assignment 




















White, non-Hispanic 


21 


14.3 




14.3 




19.0 


14.3 




38.1 


Black, non-Hispanic 


5 


0.0 


(0 


0.0 


(f) 


80.0 ** 


0.0 


(f) 


20.0 


Hispanic 


5 


60.0 


** 


0.0 


(f) 


20.0 


0.0 


(0 


20.0 


All 3 Emnlovment Barriers 




















White, non-Hispanic 


22 


4.5 




4.5 




50.0 


9.1 




31.8 


Black, non-Hispanic 


16 


31.3 


* 


0.0 


(0 


31.2 


12.5 




25.0 


Hispanic 


7 


57.1 


*** 


0.0 


(f) 


14.3 


0.0 


(f) 


28.6 



SOURCE: MDRC calculations from Background Information Form responses and from program enrollment 
and participation data from the 16 SDAs. 

NOTES: Calculations for this table are based on data for all white, black, and Hispanic male youth 
enrollees in the 18 -month study sample. Enrollees from other ethnic groups were not included because of 
small sample sizes. 

The program service categories in this table are mutually exclusive; see notes b, c, d, and e below. 

For each employment barrier subgroup, within each program service category, a chi-square test 
was applied to the difference in service receipt rates between white enrollees and black enrollees and 
between white enrollees and Hispanic enrollees. Statistical significance levels are indicated as 
* - 10 percent; ** * 5 percent; *** «■ 1 percent 

(a) The barriers to employment include: not having a high school diploma or GED at the time of 
random assignment; having worked fewer than 13 weeks during the 12 months prior to random assignment; 
and receiving AFDC or other cash assistance at the time of random assignment 

(b) The CT-OS service category includes enrollees who received CT-OS only, or CT-OS in 
combination with OJT, basic education, job search assistance, or miscellaneous program services. 

(c) The OJT service category includes enrollees who received OJT only, or OJT in combination 
with basic education, job search assistance, or miscellaneous program services. 

(d) The non-training education service category includes enrollees who received basic education 
only, or basic education in combination with job search assistance or miscellaneous program services. 

(e) The miscellaneous non-training services category includes enrollees who received miscellane- 
ous program services only, or miscellaneous program services in combination with job search assistance. 
Misce l laneous program services include such activities as assessment, job-readiness preparation, vocational 
exploration, job shadowing, and try out employment In this table, miscellaneous program services also 
include work experience, which is identified as a separate program service in other reports on the National 
JTPA Study. 

(f) A chi-square test was inappropriate because of low expected cell frequencies. 
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TABLE B. 18 



SERVICE RECEIPT RATES FOR FEMALE YOUTH ENROLLEES, 
BY EMPLOYMENT BARRIER SUBGROUP AND ETHNICITY 



Employment Barrier Subgroup (a) Sample CT-OS (b) 



Nan- Misc. 
Training JSA Non-Training 
OJT (c) Education (d) Only Services (e) 



and Ethnicity 


Size 


iCL\ 




w 








No Employment Barriers 
















White, non-Hispanic 


181 


40.9 




24.9 


6.1 


14.4 


13.8 


BiacK) non— ruspanic 












17 ft 




Hispanic 


43 


69.8 


*+* 


14.0 


0.0 (f) 


11.6 


4.7 


1 Employment Barrier 
















Worked fewer than 13 wmVs 
















in 12 months before assignment 
















White, non-Hispanic 


79 


49.4 




24.1 


3.8 


7.6 


15.2 


Black, non-Hispanic 


41 


41.5 




9.8 * 


9.8 


17.1 


22.0 


Hispanic 


43 


72.1 


** 


14.0 


2.3 


4.7 


7.0 


No high school diploma or GED 
















at assignment 
















White, non-Hispanic 


104 


30.8 




11.5 


21.2 


18.3 


18.3 


Black, non-Hispanic 




DD.D 




Id ft 






IK < 


Hispanic 


33 


69.7 


*** 


6.1 


6.1 * 


6.1 


12.1 


Receiving AFDC or other cash 
















assistance at assignment 
















White, non-Hispanic 


31 


AO A 

45.4 




zy.u 


J.Z 


U.U 


1Q A 

iy.4 


ni « TT* 

Black, non-Hispamc 


15 


53.3 




26.7 


0.0 (f) 


13.3 (f) 


0. / 


Hispanic 


9 


66.7 




0.0 (f) 


0.0 (f) 


33.3 (f) 


0.0 (f) 


2 Employment Barriers 
















Worked fewer than 13 weeks 
















in 12 months before assignment 
















and no high school diploma or GED 
















ttt tccionfnMtt 
















White, non-Hispamc 


1UO 






0 d 


3Q 

J7.Q 






Black, non-Hispanic 


74 


17.6 


* 


1.4 * 


56.8 ** 


5.4 


18.9 


Hispanic 


36 


75.0 


• 




8.3 


0.0 (f) 




Worked fewer than 13 weeks 
















in 12 months before assignment 
















raid receiving AFDC or ether cash 
















assistance at assignment 
















White, non-Hispanic 


31 


29.0 




41.9 


12.9 


3.2 


12.9 


Black, non-Hispanic 


35 


80.0 


*** 


8.6 *** 


5.7 


2.9 


2.9 


Hispanic 


20 


80.0 


*** 


5.0 ** 


0.0 (f) 


5.0 


10.0 



(continued) 
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TABLE B. 18 (continued) 



Non- Misc. 

Training JSA Non-Training 

Employment Barrier Subgroup (a) Sample CT-OS (b) OJT (c) Education (d) Only Services (e) 

and Ethnicity Size (%) (%) (%) (*) (*) 

No high school diploma or GED 
at assignment and receiving 
AFDC or other cash assitance 
at assignment 

White, non-Hispanic 23 30.4 8.7 39.1 17.4 4.3 

Black, non-Hispanic 20 20.0 10.0 25.0 5.0 40.0 ** 

Hispanic 6 33.3 0.0 (f) 16.7 16.7 33.3 * 



All 3 Employment Barriers 



'White, non-Hispanic 


52 


2S.8 


5.8 


40.4 


1.9 


23.1 


Black, non-Hispanic 


76 


32.9 


3.9 


47.4 


5.3 


10.5 * 


Hispanic 


32 


43.8 


0.0 (f) 


25.0 


6.2 


25.0 



SOURCE: MDRC calculations from Background Information Form responses and from program enrollment 
and participation data from the 16 SDAs. 

NOTES: Calculations for this table are bae*d on data for all white, black, and Hispanic £,2L*le youth 
enrollees in the 18-month study sample. Enrollees from other ethnic groups were not include*! because of 
small sample sizes. 

The program service categories in this table are mutually exclusive; see notes b, c, d, and e below. 

For each employment barrier subgroup, within each program service category, a chi-square test 
was applied to the difference in service receipt rates between white enrollees and black enrollees and 
between white enrollees and Hispanic enrollees. Statistical significance levels are indicated as 
* = 10 percent; ** « 5 percent; *** «= 1 percent. 

(»* The barriers to employment include: not having a high school diploma or GED at the time of 
random assignment; having worked fewer than 13 weeks during the 12 months prior to random assignment; 
and receiving AFDC or other cash assistance at the time of random assignment. 

(b) The CT-OS service category includes enrollees who received CT-OS only, or CT-OS in 
combination with OJT, basic education, job search assistance, or miscellaneous program services. 

(c) The OJT service category includes enrollees who received OJT only, or OJT in combination 
with basic education, job search assistance, or miscellaneous program services. 

(d) The non-training education service category includes enrollees who received basic education 
only, or basic education in combination with job search assistance or miscellaneous program services. 

(e) The miscellaneous non-training services category includes enrollees who received miscellane- 
ous program services only, or miscellaneous program services in combination with job search assistance. 
Miscellaneous program services include such activities as assessment, job-readiness preparation, vocational 
exploration, job shadowing, and try out employment. In this table, miscellaneous program services also 
include work experience, which is identified as a separate program service in other reports on the National 
JTPA Study. 

(f) A chi-square test was inappropriate because of low expected cell frequencies. 
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TABLE B.23 



ENROLLMENT RATES FOR CONTROL GROUP MEMBERS, 
BY SERVICE STRATEGY AND TARGET GROUP 



Service Strategy 


All Target 
Groups 
(%) 


Adult 
Men 

(%) 


Adult 
Women 
(%) 


Male 
Youths 
(%) 


Female 
Youths 
(%) 


Classroom Training 


3.2 


2.6 


2.6 


6.6 


3.3 


OJT/JSA 


2.4 


1.5 


2.8 


3.6 


3.7 


Other Services 


3.1 


2.5 


3.3 


2.0 


5.1 


All Strategies 


2.9 


2.0 


2.8 


3.9 


3.9 



SOURCE: MDRC calculations from Background Information Form responses and program 
enrollment data from the 16 SDAs. 

NOTE: Calculations for this table are based on data for all 5,556 control group members 
in the 18-month study sample. 
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APPENDIX C 
DATA SOURCES FOR THIS REPORT ' 



Seven data sources were used for this report Each source is described below. 

1. The JTPA Background Information Form 

A Background Information Form (BIF) was completed by all program applicants in the 
study — with assistance from local JTPA staff when necessary — no more than 45 days before 
random assignment. The BIF is the primary source of baseline information on members of the 
experimental sample (both the treatment and control groups). It includes data on the applicant's 
demographic characteristics; education, employment, and public assistance history; and family and 
household situation. Bloom, 1991, provides a complete analysis of the BIF responses and its 
implications for the National JTPA Study. 

2. Participation and Program Tracking Data from 
SPA Management Information Systems 

Each of the 16 SDAs in the study (or, in some cases, the state in which the SDA is 
located) maintains automated management information system (MIS) records that include data 
on individual JTPA enrollments, activities, and terminations. The research team requested data 
tapes from each SDA twice a year in order to compile a longitudinal record of JTPA 
participation covering the 18 months following random assignment for each sample member. 
Although each SDA collected essentially the same information, the format of their records was 
different To aid the analysis, the research team extracted the information from each SDA's 
records to be used in the study and created a standard record format for it The data elements 
used in this report include: enrollment dates, activity codes, activity start and end dates, and 
termination codes and dates. This information was used to identify initial and subsequent 
enrollments in JTPA, the number and type of activities in which enrollees participated, and the 
length of enrollment 

3. Qualitative Information and SDA Organizational Characteristics and Practices 

During the site selection process and the implementation of random assignment in the 16 
selected SDAs, the research and operations teams collected a range of information from the sites, 
including Job Training Plans for various JTPA program years, recruiting materials, lists of service 
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providers, and samples of typical vendor contracts. The team also collected information about 
the SDAs' organizational structure and client flow for the purpose of designing appropriate 
random-assignment procedures. This was done through meetings and interviews with local JTPA 
staff and administrators, service providers, local elected officials, and PIC board members. 
Systematic field research was not undertaken to study other aspects of SDA operations such as 
program planning and administrative procedures, staff attitudes and daily practices, and the nature 
of the services provided 

4. The JTPA Annual Status Report 

The JTPA Annual Status Report (JASR) provides aggregate data on the enrollment, 
participation, termination, and socioeconomic characteristics of all JTPA tenninees on an annual 
basis. It provides SDA performance data as well (see also Section 7 in this appendix). The 
research team used JASR files for program years J 987-89 to obtain background information on 
the 16 SDAs in the National JTPA Study and to compare them with all SDAs in the national 
JTPA system. 

5. The JTPA Staff Survey on Nonenrollces 

Early in the random-assignment period, it became apparent to the MDRC researchers that 
about one-third of the treatment group members were not being enrolled in JTPA activities. In 
order to learn about the extent to which SDA staff worked with nonenrollees, the research team 
chew a random sample of treatment group members from 12 of the study sites where random 
assignment was occurring. The sample was drawn during two time periods: November 1988- 
January 1989 and March-June 1989. Slightly fewer than 40 percent of these treatment group 
members were identified as not being enrolled in a JTPA activity at the time. (The final sample 
of nonenrollees used for this report includes 307 treatment group members who, according to 
MIS data, did not enroll in JTPA during the entire 18-month follow-up period.) The research 
team then conducted telephone interviews with local staff to ask about their efforts to make 
contact with the nonenrollees and to place them with a service provider or employer. 

6. JTPA Fiscal Records 

In order to compare the benefits and costs of JTPA, the research team collected data from 
program year 1968 fiscal records in all 16 SDAs. The data include the costs of administration and 
support services as well as training. In addition, all costs were tracked to identify whether money 
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was expended through fixed unit price, performance-based vendor contracts, cost-reimbursement 
contracts, or in-house accounts. The data were collected in order to permit estimates of costs 
per treatment group member for each of the three service strategies and four target groups 
studied in this report, and to separate the training costs for adults and youths. 

7. SPA Performance Data 

In general, states have the option to adapt the SDA performance standards established by 
the U.S. Department of Labor (DOL), usually by using a DOL regression model to adjust the 
standards to reflect local differences in clients and labor market conditions. There is a second 
adjustment methodology that enables states to account for other differences in local conditions 
by allowing them to include additional factors not in the DOL model. DOL must approve any 
adjustments to the standards using this second option. 

Data on SDA performance standards and levels of performance used in this report come 
from two sources. In Chapter 2, the research team used data reported in JASR to compare the 
16 study SDAs to all SDAs in the JTPA system (see Section 4 above). In Chapter 7, in an effort 
to more accurately capture trends in SDA levels of performance over the three years covered by 
the study, the research team asked the 16 SDAs to provide the final state-adjusted performance 
standards and levels for program years 1987-89, instead of relying solely on JASR data. The 
SDA-provided standards and levels were often found to be the same as those reported in JASR, 
although sometimes the state had made further adjustments that were not reflected in the most 
recent JASR file. In some cases, a comparison was not possible because the JASR data were 
missing. 
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